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A.  DICTIONARY 

Two  dictionaries  exist  for  HWIM's  travel  budget  management  domain, 
differing  primarily  in  their  number  of  lexical  items.  TRAVELDICT,  having 
1491  lexical  items  and  compatible  with  both  SMALLGRAM  and  MIDGRAM,  (Vol.  IV, 
Sec.  B)  has  been  used  extensively  throughout  the  development  of  our  system. 
BIGDICT,  having  1,138  lexical  items  and  compatible  with  BIGGRAM,  fulfills 
the  1000-word-vocabulary  requirement  set  forth  in  the  Newell  report  [Newell 
et  al . , 1973].  This  larger  dictionary,  however,  was  not  used  until  the 

final  performance  test  or  our  complete  system. 

Our  major  considerations  in  constructing  the  dictionaries  were  to 
provide  the  user  with  a habitable  and  reasonably  complete  way  of  discussing 
the  domain  and  to  provide  our  acoustic-phonetic  front  end  with  a wide  range 
of  phonetic  and  phonological  situations.  Thus,  several  methods  were  used 
in  setting  them  up. 

(1)  A set  of  domain-related  sentences  was  collected,  and  all  words  in 
these  sentences  and  in  the  sentences  from  incremental  simulations 
were  included  in  the  dictionary; 

(2)  Roget's  Thesaurus  was  used  to  find  useful  synonyms  of  existing 
words,  to  increase  the  number  of  ways  things  could  be  said; 

(3)  The  first  and  last  names  of  all  BBN  Division  <4  personnel  were 
included , 

(it)  As  were  the  names  of  all  Division  44  projects  and  their  sponsors; 

(5)  Many  place  names  (cities,  states,  countries,  universities,  etc.) 
were  added.  All  places  appearing  as  the  destinations  of  trips 
recjrded  in  the  data  base  are  represented  in  the  dictionary;  further 
place  names  were  obtained  from  meetings  announced  in  the  ACM 
calendar . 

Among  the  types  of  lexical  entries  in  these  dictionaries  are  the  root 
form  of  regularly  inflected  words,  the  root  and  inflected  forms  of 
irregularly  inflected  words,  and  a small  number  of  compound  words  like 
"New-York,"  and  "round-trip."  For  each  item,  the  dictionaries  contain  its 
pronunciation  baseform,  as  given  in  Kenyon  and  Knott's  A Pronouncing 
Dictionary  of  American  English . syntactic-semantic  class  and  feature 
information,  and,  where  appropriate,  manner  of  inflection.  The  class 
information  corresponds  to  the  categories  recognized  on  CAT  arcs  of  the 
grammar.  Thus,  beside  such  standard  classes  as  noun,  verb,  adjective, 
etc.,  the  DATE/  and  proper  NAME/  networks  of  the  grammar  recognize  such 
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classes  as  MONTH,  WEEKDAY,  FIRSTNAME,  LASTNAME,  and  CITY.  The  lexical 
items  are  well  distributed  over  the  classes,  giving  the  user  of  the  system 
a comfortable  subset  of  English  words  in  which  to  express  his  thoughts. 

The  pronunciation  base  forms  of  the  lexical  entries  are  represented  in 

the  *48  phonemes  cf  the  modified  ARPAbet,  and  four  levels  of  stress  are 

distinguished.  The  stress  indicators  are: 

!-  reduced  stress  (following  front,  back  and  retroflexed  schwas, 
syllabic  nasals  and  syllabic  [I.].; 

!0  unstressed 

! 1 auxiliary  stress 

!2  primary  stress 

Syllable  boundaries  are  marked  in  the  dictionary  with  the  symbol  *. 
This  information  is  accessed  during  dictionary  expansion  by  phonological 
rules  such  as  F16.1,  which  syllabifies  R when  it  occurs  after  a syllable 
boundary,  as  in  "arrange". 

[R16.1 

(RULE  ((«  AX  ! - * R) 

(•  AXR  ! - • 0))  ] 

(For  a complete  list  of  the  phonological  rule3  used  by  HWIM,  see  Appendix 
1.)  Inflectional  information  for  nouns,  verbs  and  adjectives  is  given  in 
terms  of  orthographic  spelling,  and  the  inflected  forms  are  then  generated 
by  rule  during  the  first  phase  of  dictionary  expansion  (Sec.  C) . For 
example,  the  lexical  item  "figure"  has  the  property  V S-D,  indicating  that 
as  a verb,  "figure"  forms  its  third  singular  present  form  orthographically 
by  adding  an  "s"  and  its  past  and  past  participle  forms  by  adding  a "d" . 

Complete  listings  of  THAVELDICT  and  BIGDICT  can  be  found  in  Appendix  5 
of  this  report. 
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B.  PHONOLOGICAL  RULES 

A necessary  consideration  in  the  design  of'  a speech  understanding 
system  is  the  incorporation  of  various  sources  of  phonological  knowledge 
[Oshika  et  al . , 1975] . It  is  well  known  that  words  can  be  pronounced  in 
continuous  speech  in  a number  of  different  ways  (depending  on  the  speaker 
as  well  as  the  linguistic  environment),  resulting  in  acoustic  signals  that 
are  sometimes  quite  different.  A speech  understanding  system  intended  to 
handle  a large  lexicon  and  multiple  speakers  will  undoubtedly  have  to 
capture  such  phonological  variations,  and  mediate  between  the  lexical 
entries  and  their  acoustic  realizations. 

It  is  also  clear  that  the  sound  of  a word  will  vary  with  it3 
phonological  environment,  such  as  the  surrounding  vowels  or  consonents  or 
the  stress  patterns,  and  that  many  of  the  sytematic  relationships  can  be 
captured  <n  general  phono1  ogical  rules.  The  development  of  such  rules, 
therefori  i .institutes  a necessary  and  intgral  part  of  speech  understanding 
research . 

Phonological  variations  in  continuous  speech  are  being  dealt  with  in 
HWIM  at  two  different  levels.  Within-word  phonological  effects  are 
accounted  for  in  a dictionary  expansion  phase  that  generates  for  each  word 
in  the  lexicon  all  its  possible  stand-alone  pronunciations.  Across-word 
phonological  effects  are  later  taken  into  account  in  the  dictionary 
compilation  procedure  of  the  Lexical  Retrieval  component  [Sec.  D]. 

1 • Within-word  Phonological  Effects 

Our  approach  to  solving  the  problem  of  within-word  phonological 
effects  can  be  characterized  as  follows:  for  each  word  in  the  dictionary,  a 
minimal  and  appropriate  set  of  lexical  base  forms  are  assumed.  This  set  of 
baseforms  is  then  augmented  via  a collection  of  generative  phonological 
rules  into  a complete  set  of  pronunciations  for  the  given  word  [Woods  and 
Zue,  1976]. 

Lexical  base  forms  are  derived  from  the  pronunciations  found  in  Kenyon 
and  Knott's  A Pronouncing  Dictionary  of  American  English . appropriately 
modified  to  reflect  the  performance  of  our  Acoustic-Phonetic  Recognition 
program.  For  example,  since  our  APR  program  distinguishes  between  front 
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and  tack  schwas,  whereas  A.  Pronouncing  Dictionary  of  American  English  does 
not,  our  base  forms  contain  both  front  and  back  schwas. 

In  the  process  of  dictionary  expansion,  a set  of  phonological  rules  is 
used  that  covers  a broad  spectrum  of  variations  ranging  from  syntactic 
inflections  (plural,  tense,  etc.)  to  vowel  reduction,  consonant, 
syllabification  and  palatalization. 

It  should  be  noted  that  the  generative  phonological  •'ules  in  the 
system  account  for  both  general  phonological  effects  and  ones  that  are 
dependent  on  the  performance  of  the  APR.  Rules  in  the  latter  category  are 
of  the  two  kinds.  Rules  of  the  first  kind  reflect  the  type  of  allophonic 
variations  distinguished  by  APR  and  are  applied  obligatorily.  For  example, 
[K's]  are  separated  into  front  and  back  [K's],  depending  on  the  features  of 
the  following  segment.  Rules  of  the  second  Kind  reflect  genuine 
shortcomings  of  the  APR  and  are  applied  optionally.  For  example,  since  the 
APR  cannot  detect  a voiceless  schwa  following  a voiceless  aspirated 
plosive,  the  schwa  is  simply  deleted  by  rule. 

An  annotated  list  of  rules  used  for  dictionary  expansion  is  included 
in  Appendix  1.  It  should  be  noted  that  this  list  represents  a subset  of 
all  the  phonological  rules  that  we  have  collected  and  developed  during  the 
course  of  our  research.  The  rules  that  we  have  chosen  for  dictionary 
expansion  reflect  either  phenomena  that  we  have  seen  in  our  acoustic  data 
or  phenomena  so  generally  common  that  we  can  expect  their  occurrence.  Some 
of  the  more  exotic  rules,  3uch  as  [r]-deletion  and  vowel  raising,  have  been 
excluded  from  the  list  so  that  the  size  of  the  expanded  dictionary  will 
remain  manageable. 

2.  Acro33-word  Phonological  Effects 

Across-word  phonological  effects  are  described  by  an  unordered  set  of 
optional  rules,  which  are  applied  to  the  complete  set  of  word 
pronunciations  that  result  from  the  consideration  of  within-word 
phonological  effects.  In  effect,  they  are  applied  to  all  word  pairs  in  the 
dictionary  that  satisfy  the  contextual  constraints.  (See  Sec.  D for  a 
fuller  discussion  of  across-word  phonological  rules.) 
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Appendix  1 also  includes  an  annotated  listing  of  a program,  written  in 
BCPL,  used  to  generate  all  acrcss-word  phonological  rules.  It  snould  be 
noted  that  although  there  exists  a large  amount  of  overlap  between  the  two 
rule  sets,  there  are  certain  rules  that  only  appear  in  the  set  describing 
across-word  phonological  effects.  For  example,  rules  are  provided  to 
Insert  a glottal  stop  between  two  vowels  across  a word  boundary,  and 
unreleased  stops  are  allowed  at  word-  and  phrase-final  positions. 
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C.  DICTIONARY  EXPANSIONS 
1 . Historical  Perspective 

In  the  Mrst  two  years  of  the  speech  project  ( 1971-73) » we  developed 
two  different  kinds  of  phon  logical  rule  mechanisms  whose  effectiveness  we 
were  able  £■•  compare.  The  first  was  a capability  to  inspect  the  segment 
lattice  of  phonetic  labeli’igs  and  apply  phonological  rules  in  reverse  to 
add  additional  branches  to  the  lattice.  The  second  was  a generative 
mechanism  to  apply  phonological  rules  in  the  forward  direction  to  the 
pronunciations  of  tne  words  in  the  lexicon  to  produce  additional  possible 
pronunciations  (i.e.,  dictionary  expansion) . We  envisioned  that  certain 
phonological  rules  would  be  best  implemented  in  one  direction  and  others  in 
the  other  direction.  In  the  current  system  (with  the  exception  of  some  of 
the  rules  used  in  the  APR  component  itself),  we  are  using  only  forward 
generative  rules.  The  reason  for  this  shift  to  exclusively  generative 
rules  is  a combination  of  the  disadvantages  of  inverse  rules  and  the 
discovery  of  an  efficient  solution  to  our  major  problem  in  the  use  of 
generative  rul«s. 

Applying  phonological  rules  in  reverse  to  the  3egmei.t  lattice  has  two 
principal  disadvantages.  First,  there  is  no  good  way  to  implement 
obligatory  rules  in  the  reverse  direction.  There  is  in  general  no  easy  way 
to  remove  from  the  segment  lattice  just  that  path  that  is  matched  by  a rule 
without  also  killing  other  paths  that  are  not  matched  by  \.>e  rule.  Even  if 
there  were  such  a method,  a path  matched  by  a rule  might  arise  accidentally 
and  not  by  the  mechanism  corresponding  to  the  rule,  and  therefore,  in  the 
reverse  direction  most  rules  must  be  considered  optional  and  not 
obligatory.  As  a result,  the  effect  of  inverse  rules  is  almost  always  to 
increase  le  subsequent  processing  required.  Secondly,  the  identification 
of  the  segments  in  the  lattice  is  or  y tentative  and  may  be  in  error. 
Hence,  many  rule  matches  are  al30  likely  . be  in  error.  Looking  at  such  a 
lattice,  one  finds  p.  'sible  sequences  of  labelings  that  one  wou1 i not 
expect  to  find  in  idealized  pronunciations,  and  consequently  the  rules  must 
be  applied  in  all  sorts  of  cases  that  they  would  not  have  faced  in  the 
generative  direction.  Moreover,  when  the  segment  lattice  includes  a score 
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for  every  possible  phoneme  at  every  point,  as  it  currently  does,  many  rules 
match  very  low  scoring  sequences,  and  it  is  difficult  to  decide  how  to 
assign  scores  to  the  additional  branches  added  to  the  lattice. 

In  the  generative  direction,  on  the  other  hand,  it  first  appears  hat 
the  acros-.-word  phonological  rules  could  not  be  applied  to  a word  unto.!  one 
knew  its  context.  Since  the  set  of  possible  sentences  that  could  be 
constructed  is  potentially  infinite,  the  complete  string  of  phonemes  to 
which  one  would  like  to  apply  rules  does  not  exist  until  a complete 
sentence  has  been  hypothesized.  Yet,  one  may  need  to  have  applied  rules  to 
a word  in  order  to  know  whether  it  ha3  occurred  in  the  input  in  the  first 
place.  The  problem  appeared  to  be  a vicious  circle.  However,  the  circle 
can  be  broken  by  a technique  developed  by  Klovstad  [Klovstad  and 
Mondschein,  1975;  Klovstad,  1976],  which  permits  across-word-boundary 
phonological  rules  to  be  applied  when  the  in-cure  dictionary  is  being 
constructed.  With  the  resulting  compiled  dictionary  structure,  all  word 
matches  (including  those  with  word-boundary  effects)  can  be  found  at  run 
time  and  the  contextual  constraints  implied  by  the  use  of  a rule  are 
automatically  propagated  to  adjacent  words.  This  technique  is  described 
more  fully  in  (Klovstad,  1976]. 

2.  Phases  of  Expansion 

The  phonological  rules  used  in  dictionary  expansion  are  divided  into 
se.  jral  different  groups  and  applied  at  several  different  phases  in  the 
process.  The  within-word  rules  are  divided  into  three  classes.  First, 
there  is  a group  of  rules  that  produce  the  pronunciations  of  regular 
inflections.  These  rules  are  used  in  an  initial  phase  of  dictionary 
expansion  that  augments  the  dictionary  with  the  regularly  inflected  forms 
of  verbs,  adjectives,  and  nouns.  They  handle  contextual  dependencies  such 
as  the  pronunciation  of  plural  endings  of  nouns  depending  on  whether  the 
preceding  phoneme  i3  voiced,  strident,  or  neither. 

A second  set  of  general  English  phonological  rules  applies  to  the 
output  of  the  inflectional  rules  to  produce  alternative  pronunciations  and 
obligatory  transformations  of  the  base  pronunciations.  These  rules 
correspond  closely  to  the  rules  that  are  recognized,  named,  and  studied  bj 
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phonologists . They  include  such  rules  as  palatalization,  syllabification, 
vowel  reduction,  glottalization , etc.,  details  of  which  are  given  in 
Section  B and  in  the  appendices  to  this  volume. 

A third  set  of  within-word  rules  handles  phonological  phenomena  that 
are  consequences  of  the  particular  capabilities  of  the  acoustic-phonetic 
recognition  component  of  the  system.  These  APR  rules  introduce 
context-specific  allophones  of  several  classes  of  phoneme  in  places  where 
the  APR  is  capable  of  making  more  precise  discriminations,  and  they 
compensate  for  such  things  as  the  inability  of  the  APR  to  detect 
nasalization  in  vowels.  The  APR  rules  thus  adjust  the  word  pronunciations 
to  make  them  more  specific  in  some  places  and  less  specific  in  other 
places,  to  match  the  discrimination  abilities  of  the  acoustic-phonetic 
front  end . 

/ 'ter  the  within-word  rules  have  been  used  for  dictionary  expansion,  a 
separate  set  of  across-word  boundary  rules  are  applied  when  the  in-core 
dictionary  tree  for  the  Lexical  Retrieval  component  is  being  constructed 
[Klovstad , 1976].  Frequently,  the  same  rule  will  apply  both  within  a word 
and  across  word  boundaries.  In  this  case,  the  rule  will  be  written  twice, 
once  in  the  format  and  conventions  for  across-word-boundary  rules,  and  once 
in  the  format  for  within-word  rules.  Table  1 shows  the  growth  of  BIGDICT 
through  dictionary  expansion.  (The  parenthesized  figure,  1097,  refers  to 
the  number  of  words  actually  recognized  by  the  BIGGRAM  grammar.' 


Roots  A irregularly  inflected  forms 
Roots  A all  inflected  forms  (words) 

BIGDICT 

1138 

1363  (1097) 

Incremental 
Expansion  Ratio 

1.20 

Pronunciation  baseforms 

1789 

1.31 

Pronunciations  following  first 
application  of  phonological  rules 

3308 

1.85 

Pronunciations  following  second 
application  of  acoustic-phonetic 
rules 

3771 

1.14 

Pronunciations  following 
application  of  across-word 
phonological  rules 

8642 

2.29 

Table  1. 
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3.  Rule  Ex panslon  System 

All  of  the  within-word  phonological  rules  are  written  in  a computer 
readable  variant  of  the  notation  that  phonologists  generally  use  for 
publication  (e.g.,  [Chomsky  & Halle,  1968]).  The  program  that  uses  these 
rules  is  an  extension  of  a program  written  by  Bobrow  and  Fraser  [1968]  for 
interactive  testing  of  phonological  rules.  That  program  provided  for  the 
definition  of  phonemes,  rules,  and  word  pronunciations  and  for  the 
expansion  of  a given  word  by  a given  list  of  rules.  It  did  not  provide 
for  the  automatic  generation  of  all  possible  pronunciations  from  a given 
set  of  optional  rules,  nor  did  it  provide  a good  way  to  systematically 
expand  all  of  the  words  in  a vocabulary  and  produce  a tabulation  of 
results . 

The  extensions  to  the  Bobrow-Fraser  system  that  we  have  implemented 
provide  for  the  specification  of  optional  rules  with  likelihood  estimates 
for  the  alternatives  of  applying  and  not  applying  them.  They  provide  for 

the  conditional  testing  of  rules  depending  on  the  success  or  failure  of 

previous  rule  matches,  the  presence  of  various  pronunciation  features 
associated  with  a pronunciation,  and  arbitrary  predicates  that  can  be 
written  in  LISP  to  test  any  desired  property  of  a word.  The  extensions 
also  provide  for  the  addition  of  pronunciation  features  to  a word 
conditional  on  the  success  of  previous  rules. 

Since  the  rule  expansion  system  has  already  been  described  thoroughly 
[Woods,  1975;  Woods  and  Zue , 1976],  we  will  not  do  so  here,  except  to 
mention  subsequent  changes  and  detail  the  terminal  instructions  required  to 
construct  the  expanded  dictionaries. 

M . Products  of  the  Expansion 

The  different  phases  of  dictionary  expansion  result  in  several 

different  products,  which  are  used  in  various  places  in  the  speech 

understanding  system.  The  initial  expansion  for  inflectional  rule 
application  is  done  primarily  to  produce  the  input  to  subsequent  phases  and 
to  produce  a VERDICT  file  which  is  used  by  the  Verification  component.  The 
VERDICT  file  contains  appropriate  base-form  pronunciations  for  all  words  in 
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the  vocabulary  — both  inflected  and  uninflected  — but  does  not  have  all 
of  the  different  pronunciations  that  the  phonological  rule  expansion  will 
later  introduce.  Many  of  these  phonological  effects  are  dealt  with 
separately  in  the  synthesis-by-rule  section  of  the  Verification  component, 
and  others  (principally  those  such  as  syllabification,  which  involve 
ambiguity  of  segmentation  into  distinct  phonemes)  are  handled  automatically 
by  the  parametric  matching  procedure. 

A^ter  the  initial  inflectional-ending  rules  are  applied,  the  main 
phonological  rule  expansion  phase  occurs  to  produce  the  various 
pronunciations  that  the  system  is  to  consider  for  each  word.  The  result  of 
this  expansion  is  an  EXPDICT  file,  which  contains  not  only  phonological 
information,  but  also  syntactic  information  used  by  the  Control  and 
Syntactic  components  of  the  system.  There  are  places  in  the  grammar  (such 
as  checking  agreement  of  the  determiners  "a"  and  "an")  where  the 
pronunciations  of  the  words  are  important  as  well  as  the  syntactic 
categories  and  semantic  markers.  The  Control  component,  on  the  other  hand, 
occasionally  wants  to  know  whether  a given  word  could  start  or  end  a 
sentence  and  may  check  syntactic  category  information  to  determine  this. 

The  final  APR-rule  expansion  is  done  to  produce  the  set  of 
pronunciations  (the  DICTTEXT  file)  that  the  Lexical  Retrieval  component 
will  use  in  constructing  its  in-core  dictionary  tree. 

Various  other  files  are  constructed  as  side  effects  of  dictionary 
expansion.  Among  these  are  EXPHONES  and  APRPHONES  files,  which  provide  a 
complete  synopsis  for  the  phonologist  of  all  the  rules  that  were  used  and 
which  words  they  applied  to,  as  well  as  the  list  of  the  rules  that 
successfully  matched  each  word.  Summary  statistics,  such  as  the  number  of 
pronunciations  before  and  after  expansion,  are  also  computed.  The  formats 
and  examples  of  the  various  dictionary  files  produced  are  given  in  Appendix 
2. 


5.  Backup  Protection  During  Expansion 

A major  new  feature  of  the  EXPFILF.  function  that  has  recently  been 
incorporated  is  an  automatic  backup  capability  to  enable  the  user  to 
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resume  a dictionary  expansion  from  a backup  sysout  in  the  event  of  a system 
crash.  This  feature  was  important  in  expanding  the  1000-word  dictionary 
since  the  amount  of  time  that  could  have  been  lost  by  a system  crash  was 
considerable.  The  backup  feature  provides  for  the  construction  of  a 
sysout  file  named  SAVED. EXPFILE  after  the  expansion  of  each  100  words.  The 
system  automatically  deletes  all  but  the  latest  two  versions  of  this  file 
each  time  it  makes  a new  one,  and  when  the  expansion  is  complete,  it 
deletes  the  remaining  ones.  If  the  computer  should  crash  during  the  run, 
the  latest  version  of  this  file  can  later  be  used  to  continue  the  expansion 
from  the  point  of  the  backup.  The  SAVED .EXPFILE  sysout  remembers  the 
positions  of  the  file  pointers  and  the  status  of  all  open  files  at  the  time 
of  the  backup  and  automatically  restores  this  state  and  continues  the 
expansion  when  it  is  run. 

Appendix  6 gives  the  necessary  details  for  using  the  dictionary 
expansion  program. 
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D.  LFXICAL  RETRIEVAL. 

Introduction 

This  section  discusses  some  aspects  of  HWIM's  Lexical  Retrieval 
component  that  have  not  been  previously  dealt  with  in  depth.  (See 
[Klovstad,  1976]  for  a detailed  description  of  the  material  taken  for 
granted  here.)  The  Lexical  Retrieval  component  is  a highly  refined  program 
for  determining  the  n most  probable  word  matches  in  a full  lexicon  or  some 
subset  of  it  appropriately  defined  by  a list  of  possible  words  and 
categories.  It  operates  on  a phonetic  segment  lattice  whose  segments  are 
described  probabilistically  (Vol.  II,  Sec.  B).  The  word  matches  that  it 
returns  are  ordered  by  probability  score  and  specify  such  information  as 
location  and  scan  direction.  The  Lexical  Retrieval  program  makes  use  of  a 
distributed  key  representation  of  the  dictionary  that  merges  common  parts 
of  different  words.  This  makes  an  effective  search  of  the  entire 
dictionary  computationally  feasible  without  having  to  consider  each  word 
separately.  The  program  provides  for  matching  words  both  left-to-right  and 
right-to-left,  taking  appropriate  account  of  across-word-bcundary 
phonological  effects  [Klovstad,  1976].  In  actual  sentence  recognition,  the 
efficiency  of  this  component  makes  it  reasonable  for  HWIM's  Syntactic 
component  to  predict  all  possible  words  and  categories  that  are  acceptable 
adjacent  to  a given  island. 

We  begin  with  a brief  review  of  our  scoring  philoscnhy,  followed  by  a 
detailed  description  of  its  implementation  at  the  acoustic-phonetic  level. 
We  then  discuss  how  Lexical  Retrieval's  matc'jr.'j  strategies  evolved  within 
the  context  of  HWIM  and  how  it  is  currently  neing  employed  there.  Finally, 
we  describe  some  tes^s  that  were  undertaken  to  evaluate  its  stand-alone 
performance . 

1 . Implementation  of  Scoring  Philosophy 

A scoring  philosophy  [Klovstad,  1976]  specifies  how  judgments  due  to 
many  different  knowledge  sources  can  be  integrated,  each  providing  its  own 
evaluation  of  a given  hypothesis  in  the  form  of  a probability.  These 
probabilities  can  then  be  multiplied  together  to  produce  a single  one  which 
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can  be  used  as  a global  measure  of  quality.  Tha  evaluation  due  to 
acoustic-phonetic  knowledge  is  of  particular  interest  for  at  least  two 
reasons : 

a)  It  has  the  biggest  effect  on  the  global  probability  measure.  This  can 
be  appreciated  by  noticing  both  the  frequency  and  magnitude  of  its 
contributions  as  specified  by  the  scoring  philosophy. 

b)  It  requires  determining  the  probability  of  a discrete  sequence  (the 
phonemes)  given  a continuous  waveform  (the  acoustics). 


This  evaluation  is  discussed  extensively  here  because  of  its  unique 
implementation  considerations.  That  is,  any  implementation  requires  both: 

a)  establishing,  for  each  phoneme  in  a sequence  to  be  scored,  a 
correspondence  with  some  segment  of  the  acoustic  waveform; 

b)  calculating  the  probablity  of  that  phoneme  sequence  given  the  waveform 
in  the  corresponding  acoustic  segments. 

These  two  requirements  parallel  the  two  major  operations  of  the  APR: 
a)  partitioning  the  acoustic  waveform  into  consecutive  temporal  segments 
and  b)  labeling  each  of  them  based  the  observed  acoustics  in  that 
segment.  As  these  operations  have  already  been  discussed  in  detail 
(Vol.  II,  Sec.  B) , we  will  focus  our  discussion  on  the  segmental  level. 

a)  Review  of  the  Scoring  Philosophy 

At  the  acoustic-phonetic  level,  our  scoring  philosophy  assigns  a score 
Si  to  a phonetic  sequence  in  proportion  to  its  probability  given  the 
acoustic  input  A (i.e.,  P(P^IA)).  In  order  to  compute  P(P^jA),  we  rewrite 
it  as  P(Pi!>nP(  A ! Pi  )/P(  A)  using  Bayes  rule  Each  word  in  our  lexicon  has 
one  or  more  pronunciations,  P^ , each  with  an  estimated  a priori 
probability,  P(Pi),  that  pronunciation  P^  will  be  used  for  this  word. 
(These  a priori  probabilities  are  computed  by  rule  during  dictionary 
expansion,  as  described  in  Section  C.)  To  rank  a word  with  pronunciation 
P^,  we  currently  calculate  P(A!Pi)/P(A)  and  report  both  it  and  PlP^)  to 
Control  as  a separate  scores.  (These  probabilities  are  kept  separate  so 
that  their  usefulness  to  the  system  can  be  investigated  at  the  discretion 
of  Control.)  P(A|P^)jP(A)  is  commonly  referred  to  as  the  likelihood  ratio. 
(In  this  context  the  words  "pronunciation"  and  "phonetic  sequence"  are  used 
interchangeably.)  Note  that  in  comparing  two  pronunciations  P^  and  Pj  that 
span  the  same  acoustics  A,  ranking  will  depend  only  on  PlAlP^  and  P(AlPj). 
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Most  often,  however,  pronunciations  span  different  parts  of  the  acoustic 
input,  so  likelihood  ratios  are  calculated. 

In  our  application,  is  rarely  a single  phoneme  but  rather  a phoneme 
sequence,  thus  posing  the  question  of  how  P(AlP^)  should  be  evaluated.  The 
Lexical  Retrieval  component  does  not  work  off  the  raw  acoustics  of  an 
utterance  but  rather  a lattice  representation  of  it  segmented  into  labeled 
regions.  Each  such  segment  has  a probabilistic  descriptor,  which  contains 
for  every  phoneme  an  estimate  of  the  probability  of  its  underlying  the 
region.  An  evaluation  of  P( A ! Pi ) then  requires  establishing  a linear 
correspondence  between  the  phonemes  in  P^  and  the  segments  associated  with 
the  acoustics  A. 

b ) Segmentation  Problems 

Unfortunately,  the  APR  occasionally  makes  segmentation  errors,  so  we 
cannot  count  on  a one-to-one  alignment  of  each  phoneme  in  the  pronunciation 
with  a segment  in  the  segment  lattice.  The  most  frequently  observed 
segmentation  errors  are  due  to  the  APR's  'missing'  a real  segment  boundary 
or  'finding'  a non-existent  one.  The  first  has  the  effect  of  producing  one 
segment  where  there  ought  to  have  been  two.  The  second  error  produces  two 
segments  where  there  ough  to  have  been  one.  Actually  these  merely  account 
for  the  first  order  effects:  'missing'  two  or  more  consecutive  real 
boundaries  or  'finding'  two  or  more  consecutive  non-existent  boundaries  are 
also  possible,  though  very  unlikely. 

c)  Path  Alignment 

In  this  section  we  describe  path  alignment  (the  alignment  of  a single 
pronunciation  against  segments  in  the  lattice)  in  terms  of  incremental 
alignments.  Three  kinds  of  incremental  alignments  - "match",  "merge"  and 
"split"  - are  involved.  A match  refers  to  the  incremental  alignment  of  a 
single  phoneme  with  a single  segment.  A merge  refers  to  the  alignment  of  a 
single  phoneme  with  two  adjacent  segments.  A split  refers  to  the  alignment 
of  two  adjacent  phonemes  with  a single  segment.  Any  path  alignment  that 
can  be  generated  by  a unique  sequence  of  incremental  alignments  is 
possible.  The  only  constraint  on  the  generation  of  path  alignments  from 
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incremental  alignments  is  that  each  phoneme  in  the  path  and  segment  in  the 
lattice  may  participate  in  only  a single  incremental  alignment. 

Although  many  other  kinds  of  incremental  alignments  are  conceivable 
( ccrresponding  to  the  higher  order  segmentation  errors  mentioned 
previously),  these  three  alignments  permit  sufficient  flexibility  to 
compensate  for  most  segmentation  errors,  since  higher  order  errors  are  very 
infrequent.  (Of  the  total  number  of  incremental  alignments  required  to 
compensate  for  the  current  APR  segmentation  behavior,  96.35  percent  are 
matches,  1.7  percent  are  merge  and  1.6  percent  are  matches.  The  remaining 
•35  percent  corresponds  to  higher  order  errors.)  When  they  do  cccur,  a 
"correct"  alignment  is  impossible,  although  a good  scoring  one  may  often  be 
obtained.  For  this  reason,  the  extra  computational  effort  required  to 
correctly  compensate  for  higher  order  segmentation  errors  does  not  appear 
to  be  justified,  and  the  possibility  of  their  occurrence  is  not  taken  into 
account. 

Figure  1 illustrates  in  terms  of  incremental  alignments  all  possible 
path  alignments  up  to  length  4.  In  this  figure,  incremental  alignments  are 
represented  as  "I",  "/\",  and  "\/"  for  match,  merge,  and  split, 
respectively,  and  path  alignments  are  pictured  as  nodes  in  the  tree. 
Alignments  of  length  n can  be  generated  incrementally  by  concatenating  one 
additional  incremental  alignment  to  its  predecessor's  alignment  of  length 
n-1 . The  predecessor's  alignment  may  be  altered  if  the  last  phoneme  was 
aligned  in  a match,  in  which  case  the  last  phoneme  may  be  realigned  as  a 
merge . 

If  there  were  no  uncertainty  in  producing  the  phoneme  to  segment 
correspondence,  the  probability  of  a given  phoneme  sequence  could  be 
calculated  directly  from  the  APR  scores.  As  we  have  just  noted,  however, 
there  is  some  uncertainty  as  to  whether  the  segmentation  produced  by  the 
APR  is  in  fact  correct.  By  correct,  we  mean  that  there  exists  some  path 
through  the  lattice  that  permits  a one-to-one  correspondence  between 
phonemes  and  acoustic  segments,  and  that  the  boundaries  between  adjacent 
segments  really  'separate'  the  acoustics  of  the  two  corresponding  adjacent 
phonemes.  Our  probabilistic  scoring  philosophy  permits  handling  this 
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uncertainty  by  considering  all  possible  path  alignments.  Specifically,  the 
probability  of  a phoneme  sequence  given  an  uncertain  segmentation  is 
defined  as  a weighted  sum  of  the  probabilities  given  each  specific  path 
alignment.  The  weight  in  each  of  these  terms  is  the  probability  of  that 
particular  path  alignment. 

To  make  things  explicit,  let  us  consider  how  P(A|P^)  would  be  computed 

given  the  phoneme  sequence  P^  = (PI  P 2 P3  P*0  and  the  segmented  acoustic 

waveform  A = (SI  S2  S3  S4).  Substituting  for  A and  P^  we  have: 

P(  A ! P± ) = P(S1  S2  S3  Si4 ! PI  P2  P3  P*0 

From  Figure  1 we  see  that  the  only  possible  path  alignments  are  those  shown 
in  Figure  2. 

By  summing  over  all  these  path  alignments  we  find  that: 

7 

P(S1  S2  S3  S4| PI  P2  P3  P4)  - Z P(S1  S2  S3  S4, alignment  i|Pl  P2  P3  P4) 

1-1 

7 

■ Z P(al Ignment  1|P1  P2  P3  P4)*P(S1  S2  S3  S4|P1  P2  P3  P4, alignment  1) 

1-1  v v / v ' 

alignment  weight  acoustic  phonetic  probability 

conditioned  on  alignment  i 


In  our  current  implementation  we  make  a simplifying  approximation,  choosing 
the  largest  of  these  terms  as  the  alignment  score.  Since  there  are  very 
few  segmentation  errors  (Vol.  II,  Sec.  B) , this  is  generally  a very  good 
approximation.  The  worst  approximation  occurs  when  there  are  severe 
segmentation  errors  present,  since  in  this  case  it  is  possible  that  two  or 
more  of  the  path  alignments  may  produce  scores  of  the  same  magnitude. 

d)  Path  Scoring 

The  scoring  philosophy  indicates  that  the  score  for  a single  path 
alignment  is  the  product  of  the  incremental  alignment  scores.  The 
incremental  score  due  to  a match  can  be  closely  approximated,  since  it  is 
provided  directly  by  the  APR  as  part  of  the  segment  description,  but 
accurate  score  approximation  for  split  and  merge  alignments  is  more 
difficult  and  is  handled  exclusively  within  the  Lexical  Retrieval 
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component.  In  order  to  estimate  these  probabilities,  we  have  created  two 
sgorinp;  matrices  ■ one  for  splits  and  one  for  merges,  based  on  labeled  split 
and  merge  alignments  found  in  our  data  base.  Because  of  the  infrequent 
occurrence  of  these  kinds  of  segmentation  errors  and  the  rather  modest 
proportions  of  our  data  base,  the  possibility  of  accurately  estimating  each 
of  the  nearly  million  split  or  merge  combinations  (915,978  for  splits  and 
723 , 3^5  for  merges)  was  clearly  out  of  the  question.  Therefore,  we  made  an 
approximation  that  resulted  in  a much  more  manageable  problem.  Our 
solution  was  to  map  the  segments  (labels)  and  phonemes  into  segment  and 
phoneme  classes  respectively,  thereby  permitting  the  creation  of 
substantially  smaller  split  and  merge  matrices  indexed  on  these  classes. 

2 • Evolution  of  Matching  Strategies 

In  matching  pronunciations  against  the  segmented  acoustic  input,  the 
need  to  accommodate  context-sensitive  word  boundary  effects,  especially 
when  context  has  not  yet  been  determined,  imposes  stringent  constraints  on 
effective  dictionary  structures  [Klovstad,  1976]  as  well  as  on  the  matching 
algorithm  discussed  above. 


This  section  describes  and  compares  three  strategies  for  handling  word 
boundary  effects  that  were  tested  during  the  development  of  the  Lexical 
Retrieval  component  a*  BBN.  The  two  special  dictionary  structures 
necessary  have  already  been  described  in  considerable  detail  [Klovstad, 
1976].  The  first  of  these  structures  was  designed  to  permit  word 
recognition  based  on  the  kernel  of  the  pronunciation.  This  is  the  part  of 
the  pronunciation  that  is  unaffected  by  word  boundary  rules.  For 
convenience  we  will  refer  to  this  as  the  Pre-wbe  structure  (pre-word 
boundary  effect).  The  second  structure  was  designed  to  permit  word 
recognition  based  on  the  kernel  plus  some  portion  of  the  word  boundary 
effect.  (The  amount  of  observed  word  boundary  effect  can  be  specified 
explicitly  in  each  word  boundary  rule.)  This  will  be  called  the  Post-wbe 
structure.  In  correspondence  with  these  two  structures,  the  three 
following  strategies  were  tried  and  tested. 


a) 


Pre-wbe  strategy 

Scans  are  made  using  the  Pre-wbe  structure  cm-.j 
consistent  adjacent  words  involves  resuming  the  eclr 
structure  where  the  prevlouS'S^af  Kf„atlc|SCa" 


g phonologically 
in  the  Pre-wbe 
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b)  Poat-wbe  strategy 

Scans  are  made  using  the  Post-wbe  structure.  Finding  phonologically 
consistent  adjacent  words  involves  resuming  the  scan  in  the  Post-wbe 
structure  where  the  previous  scan  had  terminated. 

cl  Post-wbe  strategy  with  Pre-wbe  Backtrack 

Scans  are  made  using  the  Post-wbe  structure.  During  the  scan,  when 
the  Pre-wbe  point  is  passed,  information  about  the  current  incremental 
a1 ignment  and  lattice  position  is  remembered.  Finding  phonologically 
consistent  adjacent  words  involves  backtracking  into  the  Post-wbe 
structure  to  the  Pre-wbe  position,  and  resuming  from  there. 

These  strategies  were  tried  and  tested  in  the  order  stated  above,  each 
being  an  attempt  to  lessen  the  problems  observed  in  its  predecessor. 

a)  Pre-wbe  strategy 

The  kernel  (K)  of  a pronunciation  i3  defined  with  respect  to  a given 
set  of  phonological  word  boundary  rules.  If  P is  a phonetic  spelling  and  R 
is  the  set  of  word  boundary  rules  whose  left  context  is  satisfied  by  P, 
then  the  kernel  K is  defined  as  the  longest  left  substring  of  P that  would 
be  unchanged  by  the  application  of  any  of  these  rules  R. 

A Kernel  tree  is  formed  by  merging  common  initial  phoneme  sequences  of 
word  pronunciation  kernels  such  that  each  path  in  the  tree  specifies  a 
unique  phonetic  sequence.  As  a result,  no  two  distinct  paths  through  the 
tree  ere  associated  with  the  same  phonetic  sequence,  and  no  two  distinct 
initial  phonetic  sequences  are  associated  with  the  same  path.  A 
pronunciation  path  is  a path  that  terminates  on  a node  in  the  tree  at  which 
some  word  is  identified. 

After  the  Kernel  tree  has  been  completed , additional  paths  must  be 
added  to  permit  the  transformations  specified  in  the  rules  and  to  complete 
the  kernels  (if  necessary).  For  every  pronunciation,  an  entry  tree  is 
constructed  containing  just  these  necessary  paths.  One  entry  tree  can  be 
hared  by  different  pronunciations  that  satisfy  the  same  set  of  word 
boundary  rules. 

Each  entry  tree  contains  one  path  for  the  kernel  residue.  This  is  the 
portion  of  the  pronunciation  that  can  be  changed  by  a word  boundary  rule. 
(The  original  pronunciation  can  be  generated  by  concatenating  the  kernel 
and  the  kernel  residue.)  At  the  node  terminating  this  path,  there  is  a 
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branch  that  enters  the  root  of  the  Kernel  tree.  The  other  path's  provide 
for  the  transformations  in  the  rule  subset.  At  each  node  that  completes  a 
transformation,  there  is  a branch  that  enters  the  Kernel  tree  at  a node 
determined  by  the  right  context  of  the  corresponding  rule.  The  set  of 
words  that  satisfy  its  right  context  are  precisely  those  words  that  start 
with  the  specified  phonetic  sequence.  The  set  of  words  is  then  uniquely 
specified  by  the  node  in  the  tree  that  terminates  the  right  context,. 

Now  by  concatenating  these  pronunciations  - continuing  each  kernel  by 
following  paths  in  the  corresponding  entry  tree  - one  observes  that  all 
sequences  of  pronunciations  and  possible  word  boundary  effects  can  be 
generated.  A more  detailed  description  with  examples  is  available  in 
[ Klovstad , 1976]. 

These  kernel  and  entry  trees  constitute  the  necessary  structures  for 
the  Pre-wbe  strategy.  In  this  strategy,  scans  for  word  matches 
phonological ly  compatible  with  a given  word  match  are  begun  using  the 
specially  constructed  entry  tree  associated  with  each  pronunciation  in  the 
kernel  tree.  In  such  a scan,  the  path  necessary  for  kernel  completion  is 
considered  along  with  all  and  only  those  paths  that  correspond  to 
compatible  word  boundary  rules.  All  scans,  whether  started  at  the  root  of 
the  Kernel  tree  as  an  initial  scan  or  at  the  root  of  a particular  entry 
tree  as  an  anchored  scan,  stop  matching  a word  when  its  kernel  is  completed 
(i.e.,  the  Pre-wbe  point  in  one  of  its  pronunciations  is  reached). 
Continued  scans  from  this  point  (that  is,  the  root  of  the  kernel's  entry 
tree)  for  compatible  matches  are  possible. 

There  are  two  main  advantages  to  the  Pre-wbe  strategy: 

a)  The  Pre-wbe  structure  is  less  complex  and  requires  less  memory  than 
the  Post-wbe  structure 

b)  It  is  a less  complex  and  more  efficient  search  strategy  than  the 
others . 


However,  its  performance  is  deficient  with  respect  to  the  following 
two  problems: 

a)  waS2rrhCLbut  P°orly  scoring  word  would  be  eliminated  when  this  could 
Prevented  by  the  high  score  of  some  robust  phoneme  outside 
its  kernel,  (i.e.,  beyond  the  Pre-wbe  stopping  point) 
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b)  An  Incorrect  word  would  outscore  correct  words  when  none  of  the  wbe  or 
kernel  completion  paths  would  have  scored  well  and  would  have  lowered 
the  overal 1 score . 

These  problems  are  generally  more  severe  for  short  words,  since  the 
unscored  phonemes  are  more  likely  to  be  a significant  portion  of  the  total 
word.  Both  problems  were  observed  occasionally  and  were  at  least  partially 
responsible  for  the  unsuccessful  recognition  of  some  sentences. 

b)  Post-wbe  strategy 

It  was  believed  that  scoring  phonemes  beyond  the  end  of  the  kernel 
would  eliminate  both  of  the  above  problems,  and  the  Kernel  tree  with 
embedded  word  boundary  rules  was  used  as  a guide  in  the  consideration  of  an 
alternative  structure. 

In  this  new  structure  the  distinction  between  Kernel  and  entry  tree  is 
blurred  somewhat,  for  the  transition  from  one  to  the  other  is  now 
recognizable  only  by  special  marks.  Now  if  a specially  marked  node  is 

reached  while  scoring  in  the  Kernel  tree,  information  that  identifies  the 
particular  word  and  pronunciation  being  scored  is  picked  up  and  saved,  and 
scoring  continues  on  into  the  appropriate  entry  tree.  At  some  node  in  the 
entry  tree,  a second  and  different  mark  indicates  that  scoring  had 
proceeded  far  enough.  Kernel  paths  that  represent  complete  pronunciations 
end  at  unmarked  nodes  in  the  Kernel  tree.  With  this  strategy  two  word 
matches  were  said  to  be  equivalent  only  if  they  spanned  identical  portions 
of  the  segment  lattice  (i.e.,  their  starting  and  stopping  segments  are 
identical),  they  both  reference  the  3ame  word  (i.e.,  the  same  orthographic 
spelling)  , and  they  both  continue  on  from  the  same  node  in  the  transition 
tree  (same  word  boundary  effects).  Our  experience  with  this  strategy 
confirmed  its  anticipated  defects: 

a)  The  list  of  words  returned  on  each  scan  tends  to  contain  many  variants 
of  essentially  the  same  word  match,  the  only  difference  being  possible 
word  boundary  effects. 

b)  The  left  and  right  context  specifications  in  the  word  boundary  rules 
are  not  accurate  enough,  being  either  too  narrow  or  too  broad.  In  the 
former  case,  the  boundary  effect  on  an  otherwise  correct  word  match 
would  be  incompatible  with  the  word  that  actually  adjoined  it.  In  the 
latter,  an  incorrect  word  would  be  permitted  adjacent  to  a correct 
word  by  virtue  of  too  lax  a specification. 
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c)  Post-wbe  strategy  with  Pre-wbe  Backtrack 

Although  the  Post-wbe  strategy  eliminated  the  deficiencies  noted  for 
the  Pre-wbe  strategy,  it  introduced  many  new  word  match  variants,  thereby 
reducing  system  efficiency.  To  eliminate  some  of  these  word  matches 
variants,  we  changed  our  concept  of  word  match  equivalence.  Remembering 
that  there  were  fewer  word  match  variants  when  the  scoring  stopped  at  the 
kernel,  we  decided  to  define  equivalence  with  respect  to  kernel  word 
matches  as  we  had  in  the  Pre-wbe  strategy.  We  continue  to  score  beyond  the 
kernel  as  in  the  Post-wbe  strategy,  but  we  pick  up  and  save  (in  addition  to 
word  and  pronunciation  information)  both  score  and  boundary  information 
when  we  reach  the  specially  marked  node  indicating  the  end  of  the  kernel. 
We  now  use:  a)  the  kernel  boundary  in  place  of  the  stopping  segment  for 
determining  spanning  equivalence,  and  b)  the  kernel's  entry  tree  in  place 
of  the  stopping  node  in  the  transition  tree  for  determining  word  boundary 
rule  equivalence.  Since  we  now  save  the  kernel  boundary,  we  are  able  to 
start  each  subsequent  adjacent  word  scan  at  the  root  of  the  kernel's  entry 
tree.  Thus  word  match  equivalence  is  defined  as  in  the  Pre-wbe  strategy, 
word  matches  are  scored  as  in  the  Post-wbe  strategy,  and  adjacent  word 
scans  are  done  as  in  the  Pre-wbe  strategy. 

Each  word  match  that  extends  beyond  its  kernel  now  has  two  scores,  one 
up  to  the  end  of  the  kernel  and  a second  representing  the  entire  match. 
Individual  word  matches  are  ranked  by  their  entire  scores.  The  score  of 
two  consecutive  word  matches,  however,  must  not  be  computed  from  their 
entire  scores,  because  that  would  constitute  scoring  the  region  beyond  the 
kernel  twice.  In  order  to  allow  the  Control  component  to  account  for  this 
overlap  of  adjacent  word  matches  as  it  builds  up  multiword  theories,  the 
Matcher  sends  up  both  entire  and  overlap  scores.  Although  performing  two 
consecutive  scans  with  this  strategy  requires  more  computation  than 
previously,  because  paths  in  the  entry  tree  are  scored  twice,  word  match 
reliability  is  substantially  increased.  Its  efficiency  and  performance  at 
the  system  level  is  better  than  either  of  the  alternative  strategies. 
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8.  Use  of  Lexical  Retrieval  in  HWIM 


During  the  development  of  HWIM's  Lexical  Retrieval  component,  certain 
activities  were  codified  that  appeared  especially  useful  within  the  system 
environment : 


a)  Global  Scan 

A sliding  scan  ir:  both  directions  ( left-to-right  and  right-to-left) 
done  to  establish  a maximum  score  profile  for  shortfall  scoring  (see 
Sec.  E,  this  volume),  and  in  some  cases  to  find  seed  words.  If  some 
region  of  the  lattice  is  not  suitably  covered,  subsequent  scans  are 
made  in  the  vicinity  of  the  uncovered  region. 

b)  Left  End  Words  Scan 


A sequence  of  scans  done  at  all  possible  left  boundaries  of  the 
utterance  looking  for  syntactically  allowable  initial  words  and  then 
"near"  the  left  end  (as  specified  by  Control)  looking  for  possible 
seed  words. 


c)  Word  Anchored  Scans 

Scans  done  in  the  same  direction  as  the  word  match  with  which  they 
are  to  be  phonologically  compatible.  (Sometimes  Control  has  to  make  a 
very  restricted  scan,  looking  only  for  a word  match  going  in  the 
direction  in  which  a subsequent  extension  is  to  be  made.)  The  purpose 
of  these  scans  is  to  make  a phonologically  consistent  extension  of  the 
current  theory  in  the  given  direction  as  specified  by  the  word 
boundary  rules. 

d)  End  Scan 


A scan  used  only  for  the  Post-wbe  strategy  with  Pre-wbe  backup.  Its 
purpose  is  to  determine  the  best  way  to  complete  a utterance.  This 
requires  scoring  all  word  boundary  effects  (as  if  doing  an  anchored 
scan  from  one  specific  word)  and  their  associated  ending  penalties  and 
then  picking  the  best  combination  for  use  in  scoring  the  complete 
utterance.  I Ending  penalties  are  determined  by  the  APR  component  for 
each  boundary  in  the  segment  lattice.) 


4 . Performance 


We  feel  we  have  developed  both  an  efficient  and  an  effective  method  of 
lexical  retrieval  for  use  in  general  speech  understanding  systems.  In 
order  to  demonstrate  its  stand-alone  performance,  it  was  tested  on  the  same 
set  of  sentences  that  were  used  to  evaluate  the  entire  speech  system.  All 
entries  in  the  expanded  BIGDICT  dictionary  (1362)  were  possible  candidates 
ana  only  correct  inflections  were  counted  as  correct.  The  performance  is 
given  for  two  different  types  of  operation:  a)  sliding  scans,  and  b) 
anchored  s^ans. 
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a)  Sliding  Sean  Performance 

In  this  test,  the  Lexical  Retrieval  component  scanned  the  segment 
lattice  of  each  utterance  for  all  dictionary  words.  A word  match  was 
counted  as  correct  if  it  was  a word  in  the  utterance  and  was  found  in  its 
proper  place  in  the  segment  lattice.  Although  the  number  of  words  per 
sentence  varied  considerably,  we  made  no  attempt  to  compensate  by 
requesting  a proportional  number  of  word  matches.  Instead,  Lexical 
Retrieval  was  directed  to  return  the  15  most  probable  word  matches  for  each 
sentence.  Figure  3 shows  one  performance  measure:  the  average  ratio  of 
correct  to  incorrect  words. 


Average  Number  of  Words  per  Sentence  

Average  Number  of  Distinct  Word  Matches  . , . 
Average  Number  of  Correct  Words  per  Scan  . . . 
Average  Number  of  Incorrect  Words  per  Scan  . . 
Average  Ratio  of  Correct  to  Incorrect  Words  , 


6.20 

12.30 

2.17 

10.13 

.2142 


Fig.  3.  Performance  measurement. 


Another  performance  measure  derived  from  these  same  scans,  the  ability 
of  Lexical  Retrieval  to  distinguish  between  correct  and  incorrect  words  is 
presented  in  Figure  ii.  Here  the  rank  of  the  highest  scoring  correct  word 
is  plotted  as  a histogram. 
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b)  Anchored  Scan  Performance 

A more  meaningful  performance  measure  within  the  context  of  the  whole 
system  tested  how  well  Lexical  Retrieval  performed  when  looking  adjacent  to 
the  correct  word  in  the  correct  place  in  the  segment  lattice.  These  scans 
were  done  as  above  with  absolutely  no  selectional  (i.e.,  syntactic  class 
or  word)  constraints.  We  started  at  the  beginning  of  each  utterance  and 
scanned  for  all  words  in  the  dictionary.  Since  only  the  top  15  positions 
were  going  to  be  used  explicitly,  scans  were  made  with  a short  completion 
stack  [Klovstad,  1976].  Occasionally,  a second  (selective)  scan  was 
necessary  because  the  correct  word  would  not  be  returned  as  one  of  the  top 
scoring  15.  Once  the  correct  word  was  found,  an  anchored  scan  was  made  to 
its  right.  This  procedure  was  continued  until  every  word  in  the  utterance 
had  been  found.  In  Figure  5,  the  rank  (in  the  completion  stack)  of  the 
correct  word  is  plotted  as  a histogram.  In  this  same  figure,  a cumulative 
distribution  of  these  results  is  also  presented.  Almost  all  the  "correct" 
words  that  were  not  among  the  top  15  were  the  result  of  noticeably  bad 
segmentations  in  the  lattice.  For  each  anchored  scan  (321  were  made  in 
all),  the  score  difference  between  the  correct  word  and  the  highest  scoring 
incorrect  word  was  recorded.  (N.B.  This  difference  is  positive  only  when 
the  correct  word  appears  in  position  1.)  The  histogram  of  these  score 
differences  is  shown  in  Figure  6. 
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E.  CONTROL  STRATEGY 

1 • First  Attempt s at  A Control  Strategy 

In  our  original  speech  understanding  system,  SPEECHLIS,  [Woods,  1974; 
Rovner  et  al . , 1974],  the  control  strategy  was  an  implementation  of  the 
perceptual  strategy  described  in  Vol.  I,  Sec.  B in  its  full  generality.  An 
initial  scan  of  the  segment  lattice  by  the  Lexical  Retrieval  component 
looking  for  robust  cues  was  done  to  form  initial  one-word  theories 
(stimulus  driven  hypothesization) . Each  such  theory  forced  a call  on  a 
Semantic  component  to  set  semantic  monitors,  make  proposals  and  detect 
events.  The  events  and  theories  resulting  from  this  semantic  processing 
consisted  of  non-overlapping  collections  of  semantically  related  words 
(generally  separated  by  gaps).  When  the  top-ranking  theory  became 
semantically  "complete",  at  least  with  respect  to  the  contents  of  the  word 
lattice,  a Syntactic  component  was  called  upon  to  evaluate  it  and  propose 
words  to  fill  the  gaps. 

The  SPEECHLIS  system  contained  separate  queues  of  events,  notices,  and 
proposals,  and  was  used  experimentally  to  explore  such  issues  as:  when 

should  proposals  be  done  as  opposed  to  doing  another  event,  when  should 
Syntax  be  called  to  evaluate  a theory  and  make  proposals,  when  should 
Semantics  be  called,  and  how  dense  should  the  initial  word  lattice  be. 
This  system  was  the  culmination  of  our  incremental  simulation  approach  to 
speech  understanding,  and  we  learned  a lot  from  it.  As  a result  of  our 
experience  with  this  system,  we  evolved  a more  specialized  framework  for 
our  second  generation  speech  understanding  system  (HWIM),  in  which  we 
embodied  our  best  intuitions  on  these  issues. 

2.  Island-driven  Strategies 

The  type  of  strategy  that  appeared  most  effective  as  a result  of  our 
experience  with  SPEECHLIS  is  one  that  we  have  termed  Island  driven . This  is 
a strategy  in  which  all  theories  consist  of  a contiguous  sequence  of  words 
with  no  gaps  between  them  (an  island ) . 

In  the  SPEECHLIS  strategy,  a theory  generally  contained  several 
semantically-related  islands  separated  by  gaps.  The  reason  for  grouping 
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semantical  1 y-related  islands  into  a single  theory  was  the  assumption  that 
the  semantic  hypothesis  that  created  such  a theory  could  affpct  the 
syntactic  processing  of  the  individual  islands.  However,  we  failed  to  find 
useful  ways  to  capitalize  on  this  possibility,  and  in  actuality,  the 
syntactic  processing  of  a given  island  was  almost  totally  independent  of 
other  islands  in  the  theory.  (The  only  significant  place  where  this  was 
not  true  was  when  two  islands  grew  close  enough  together  that  the  gap 
between  them  could  be  filled  by  a single  word.  In  this  case,  the  Syntactic 
component  took  them  both  into  account,  making  strong  proposals  in  order  to 
try  to  fill  the  gap.)  On  the  other  hand,  the  multiple  island  theories  had 
an  associated  cost  in  processing  due  to  the  possibility  of  the  same  island 
occurring  in  several  different  theories  (i.e.,  theories  that  differed  in 
the  words  they  hypothesized  at  other  points  of  the  utterance) . Although 
care  was  taken  that  a given  island  need  be  syntactically  evaluated  only 
once,  independent  of  how  many  theories  it  occurred  in,  a major  portion  of 
the  Syntactic  component's  overhead  in  SPEECHLIS  lay  in  attempting  to  ke^p 
track  of  islands  thjt  had  previously  been  parsed  and  recognizing  them  when 
they  appeared  again  in  new  theories.  Thus,  in  handling  multiple  island 
theories,  we  were  carrying  a burden  for  which  we  were  deriving  at  most  a 
marginal  theoretical  advantage. 

It  should  be  noted  here  that  a continuing  goal  in  our  search  for 
effective  speech  understanding  strategies  has  been  to  find  ways  for  having 
what  is  found  at  one  point  in  the  utterance  affect  how  we  analyze  other 
portions  of  the  utterance.  In  most  cases,  syntactic  information  can  only 
influence  word  hypotheses  immediately  adjacent  to  an  island.  The 
techniques  explored  in  SPEECHLIS  were  an  attempt  to  discover  viable  ways  to 
get  more  global  effects  to  lend  support.  However,  while  the  semantic 
intersection  technique  does  permit  one  to  detect  coincidences  between 
widely  separated,  but  semantically  related,  words,  the  way  that  this 
information  was  coupled  into  the  Control  and  Syntactic  components  in 
SPEECHLIS  did  not  provide  sufficient  support.  A more  effective  way  to  tap 
this  semantic  intersection  information  might  have  been  to  use  it  in  the 
priority  scoring  of  theories,  without  explicitly  forming  combinations  of 
separated  islands.  No  further  work  was  done  in  this  direction,  however. 
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3.  Differenced  Between  SPEECHLIS  and  HWIM  Control  Frameworks 

In  our  second  generation  system,  HWIM,  various  features  of  the  general 
perceptual  strategy  described  Vol.  I,  Sec.  A were  particularized, 
surrendering  some  flexibility  for  decreased  overhead.  On  the  other  hand, 
we  have  implemented  in  HWIM  a variety  of  flag-governed  options,  so  that  the 
effects  ~f  different  control  strategies  could  be  explored.  HWIM  still 
represents  only  a sample  point  in  an  evolutionary  process,  and  it  still 
contains  implementation  overheads  such  as  these  flags  which  have  been 
designed  into  it  to  promote  flexibility  for  experimentation. 

In  both  SPEECHLIS  and  HWIM,  an  initial  scan  of  some  portion  of  the 
utterance  is  used  to  form  initial  one-word  seed  events,  but  beyond  this 
point,  the  two  systems  differ  considerably.  In  HWIM,  an  event  is  processed 
by  giving  it  to  the  Syntax  component  which  combines  syntactic,  semantic, 
pragmatic,  and  prosodic  information  (see  Vol.  IV)  to  (a)  determine  whether 
the  resulting  theory  could  be  part  of  a complete  sentence;  (b)  produce 
proposals  for  adjacent  words  and/or  categories;  and  (c)  possibly  adjust  the 
score  of  the  resulting  theory  to  reflect  syntactic,  semantic,  pragmatic,  or 
prosodic  information.  Whereas  in  SPEECHLIS  we  had  the  option  of  doing 
several  events  before  actively  pursuing  proposals,  as  well  as  that  of 
monitoring  all  predictions  while  only  actively  proposing  some  of  them,  in 
HWIM  we  propose  all  predictions.  Moreover,  where  SPEECHLIS  gave  Syntax  the 
option  of  doing  only  a partial  evaluation  of  a theory,  in  HWIM  the 
Syntactic  component  does  all  its  processing  the  first  time  it  is  called, 
making  all  possible  compatible  predictions  at  that  time.  This  is  because 
our  experience  with  SPEECHLIS  did  not  suggest  any  effective  criteria  for 
deciding  when  to  call  Syntax  back  for  additional  processing  and 
predictions.  Nor  did  it  suggest  a good  criterion  for  calling  for  further 
unconstrained  word  scans  in  a portion  of  an  utterance.  As  a result, 
monitors  set  without  explicit  proposals  were  only  triggered  by  words  from 
the  initial  scan  or  words  independently  proposed  and  serendipitousl y found. 

Another  major  difference  between  the  two  systems  is  the  short 
circuiting  in  HWIM  of  much  of  SPEECHLIS's  monitor-propose-notice  activity. 
Whereas  SPEECHLIS  permitted  a number  of  events  to  be  processed  before  any 
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of  their  proposals,  in  HWIM,  proposals  are  done  immediately  as  part  of  the 
processing  of  the  event.  Also,  where  SPEECHLIS  used  monitors  to  catch  the 
words  that  came  back  from  Lexical  Retrieval  and  put  them  together  with  the 
theories  that  proposed  them,  in  HWIM  there  is  only  one  proposing  theory  for 
each  batch  of  proposals.  This  means  that  appropriate  notices  can  be 
constructed  directly,  eliminating  the  need  for  word  lattice  monitors  (as 
well  as  one  of  the  reasons  for  having  a word  lattice  around!).  This  saves 
both  the  memory  that  had  been  required  for  storing  these  monitors  and  also 
considerable  processing  time. 

Monitors  remain  in  the  H*»IM  system  only  for  detecting  island 
collisions . where  two  theories  notice  the  same  word  from  opposite 
directions.  Whenever  an  event  is  created  to  add  an  additional  word  to  a 
theory,  an  island  table  is  consulted  to  see  if  the  same  word  has  been 
noticed  from  the  other  direction.  If  so,  a collision  event  can  be  created 
that  will  combine  the  two  theories  and  the  noticed  new  word  into  a single 
new  theory.  Each  entry  in  an  island  table  is  effectively  a monitor  that  is 
watching  for  a theory  to  notice  a particular  word  in  a particular 
direction . 

Other  differences  between  HWIM  and  SPEECHLIS  derive  from  (a)  solutions 
to  problems  that  had  not  yet  been  faced  in  SPEECHLIS;  (b)  the  development 
of  new  knowledge  sources  (such  as  a synthesis-by-rule  program  and 
parametric  matcher  - Vol.  II,  Sec.  C and  D)  and  new  versions  of  existing 
ones  (such  as  a new  parser  and  "pragmatic"  grammar  - Vol.  IV,  Sec.  A and 
B);  and  (c)  the  systematization  of  a uniform  scoring  philosophy  (Vol.  I, 
Sec.  B) . 

Under  category  (a)  belongs  the  concept  of  rectification . In 
SPEECHLIS,  we  developed  the  concept  of  a fuzzy  word  match  to  accommodate 
finding  several  matches  of  the  same  word  at  approximately  the  same  place, 
but  with  different  end  points  and  different  scores.  (This  was  due  to 
several  factors:  different  word  boundary  effects,  branching  in  the  segment 
lattice,  and  split  and  merge  alignments  in  the  Lexical  Retrieval  component 
(see  Vol.  Ill,  Sec.  D) . However,  we  did  not  have  a facility  for  checking 
the  compatibility  of  word  pairs  from  adjacent  fuzzy  word  matches.  Such  a 
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facility  would  allow  us  to  assign  a "rectified  score"  tc  a theory  which 
would  take  account  of  the  amount  by  which  the  score  of  the  best  such  pair 
falls  below  the  sum  of  the  scores  on  the  best  members  of  the  two  fuzzies. 
Thus,  in  SPEECHLIS,  adjacent  fuzzy  word  matches  were  given  scores  that 
might  have  been  too  lenient  where  adjacent  word  compatibility  was 
concerned.  In  HWIM,  techniques  were  developed  to  check  this  compatibility 
and  assign  scores  accordingly. 

4.  Priority  Scoring 

The  score  assigned  to  a theory  by  the  summation  of  lexical  retrieval 
scores  (essentially  the  log  probability  of  its  words  being  correct)  «e 
refer  to  as  the  quality  score  of  the  theory.  We  distinguish  this  from  a 
possibly  separate  score  called  the  priority  score . which  is  used  to  rank 
events  on  the  event  queue  to  determine  the  order  in  which  they  are  tc  be 
done.  In  early  versions  of  HWIM,  we  used  the  quality  score  itself  as  the 
priority  score.  However,  we  have  developed  several  algorithms  with 
interesting  theoretical  properties  using  priority  scores  that  are  derived 
from  but  not  identical  with  the  quality  score. 

The  first  3uch  score  is  a measure  of  the  difference  between  the 
particular  quality  score  for  a theory  and  an  upper  bound  on  the  possible 
quality  score  for  any  theory  covering  the  same  portion  of  the  utterance. 
We  call  this  difference  the  shortfal 1 score . and  it  can  be  shown  that  using 
the  shortfall  score  as  a priority  score  under  appropriate  conditions 
guarantees  finding  the  best  scoring  interpretation  of  the  input  utterance 
[Woods,  1976].  Using  the  quality  score  it3elf  as  a priority  score  does  not 
guarantee  this.  Other  priority  scores  are  obtained  by  dividing  either  the 
quality  score  or  the  shortfall  score  by  the  time  duration  of  the  island  to 
give  quality  density  and  shortfall  density  scoring , respectively.  Since  we 
have  previously  given  a fairly  complete  derivation  of  the  shortfall  and 
shortfall  density  scoring  strategies  together  with  proofs  of  their 
completeness  [Woods,  1976],  we  will  instead  present  here  a brief 
recapitulation  of  the  different  strategies. 
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5.  Shortfall  Scoring 

As  mentioned  above,  one  of  the  kinds  of  priority  scores  that  can  be 
used  for  ordering  the  events  on  the  event  queue  is  a shortfall  score.  This 
score  measures  the  amount  by  which  the  score  of  a theory  falls  below  an 
upper  bound  on  the  possible  score  that  could  be  achieved  on  the  same 
region.  When  shortfall  scoring  is  being  used,  a MAXSEG  pi  ofile  is 
constructed  having  the  property  that  the  score  of  a word  match  between 
boundaries  i and  j will  be  less  than  or  equal  to  the  area  under  the  MAXSEG 
profile  from  i to  j . (The  latter  is  called  tne  MAXSCORE  for  the  region  from 
i to  j.)  The  shortfall  score  for  a theory  is  then  computed  as  the  sum  over 
all  the  word  matches  in  the  theory  of  the  difference  between  the  score  of 
the  word  match  and  the  MAXSCORE  for  the  same  region.  This  score  is  always 
negative,  although  only  the  magnitude  is  printed  in  the  system  traces  (e.g. 
Vol.  I,  Appendix  2).  The  preferred  theory  is  the  one  with  the  smallest 
absolute  shortfall. 

A MAXSEG  profile  is  constructed  incrementally  as  follows:  Whenever 
the  score  of  a word  match  exceeds  the  MAXSCORE  for  the  region  it  covers, 
the  excess  score  is  distributed  over  the  region  to  raise  its  MAXSCORE  to 
equal  the  word  match  score.  While  the  way  in  which  this  excess  score  is 
distributed  does  not  affect  the  theoretical  claims  of  the  algorithm,  it  is 
desirable  to  do  it  in  such  a way  as  to  minimize  the  amount  by  which  the 
shortfall  of  other  words  that  overlap  the  region  is  raised.  Our  current 
algorithm  is  to  distribute  the  excess  score  over  the  segments  covered  by 
the  word  match  that  are  not  already  bounded  by  the  profile  and  to  divide  it 
proportional  to  the  durations  of  the  segments.  Oth^r  distribution 
algorithms  are  possible,  some  of  which  have  been  tried.  This  one  is  better 
than  some,  but  there  are  probably  better  strategies  to  be  found.  Keeping 
the  MAXSEG  profile  as  low  as  possible  while  still  satisfying  the  upper 
bound  condition  is  important,  since  excessively  conservative  upper  bounds 
translate  directly  into  an  unnecessary  increase  in  the  breadth-first  nature 
of  the  search,  requiring  more  events  to  be  processed  before  finding  the 
chosen  interpretation. 
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The  theoretical  characteristic."  of  the  shortfall  scoring  algorithm  are 
such  that  if  the  words  are  returned  by  the  Lexical  Retrieval  component  in 
shortfall  order  and  events  are  processed  in  order  of  increasing  absolute 
shortfall  (plus  a few  other  assumptions,  documented  in  [Woods,  1976]),  then 
the  first  complete  spanning  interpretation  iound  will  be  the  best  scoring 
in'.  . pr*-  ation  that  can  be  found  by  any  strategy.  We  refer  to  this 
conditio  as  "completeness”.  For  speech  understanding  applications, 
completeru..  is  a desirable  property,  but  not  necessarily  essential  if  the 
cost  of  its  attainment  is  too  great.  Shortfall  scoring  has  the  property  of 
being  complete  without  searching  the  entire  space.  Proof  of  its 
completeness  depends  onl  on  the  fact  that  when  the  first  complete  spanning 
theory  is  found,  all  other  events  on  the  queue  will  already  have  fallen 
below  the  ideal  maximum  score  by  a greater  amount.  Thus  the  result  does  not 
depend  on  the  scores  being  likelihood  ratios,  nor  does  it  make  any 
assumption  about  the  nature  of  the  grammar  (e.g.,  that  it  be  a finite  state 
Markov  process)  , provided  a parser  exists  that  can  make  the  necessary 
judgments.  The  completeness  also  does  not  depend  on  the  order  of  scanning 
the  utterance  — it  is  satisfied  both  for  middle-out  arid  for  left-to-right 
strategies . 

The  actual  implementation  of  the  algorithm  in  the  HWIM  system  does  not 
satisfy  the  theoretical  prerequisites  exactly,  since  the  words  are  not 
necessarily  returned  by  the  Lexical  Retrieval  component  in  shortfall  order. 
However,  a close  approximation  to  the  theoretical  algorithm  is  achieved  by 
retrieving  words  in  relatively  large  batches  and  ordering  them  in  shortfall 
order  afterwards.  An  initial  scan  of  the  utterance  is  done  to  determine  an 
i itial  estimate  of  the  MAXSEG  profile  (as  well  as  to  "ind  seed  words),  and 
this  estimate  is  relatively  stable  thereafter.  However,  occasionally  a 
word  match  is  found  in  the  course  of  an  analysis  that  incre  es  the  MAXSEG 
profile  In  some  region,  ar.d  in  that  case  all  of  the  events  whose  scores 
could  be  affected  are  rescored  and  repositioned  in  the  event  queue.  If  a 
Lexical  Retrieval  component  designed  to  return  words  in  shortfall  order 
were  available,  this  would  not  be  the  case,  and  the  initial  upper  bound 
would  i omain  stable  throughout  the  analysis  (such  a retrieval  algorithm 
would  have  to  dete  mine  a true  uppar  bound  in  the  course  of  its  scan). 
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6 . Density  Scoring 

Another  type  of  priority  scoring  is  density  scoring.  Here  the  score 
used  to  order  the  event  queue  is  some  basic  score  divided  by  the  duration 
of  the  event.  Conceptually,  we  can  think  of  this  priority  scoring  metric 
as  predicting  the  potential  score  for  the  region  not  covered  by  a theory  to 
be  an  extrapolation  of  the  same  score  density  already  achieved.  (In  these 
terms,  the  shortfall  strategy  can  be  thought  of  as  predicting  that  tne 
upper  bound  will  be  achieved  in  the  uncovered  region.)  Unlike  the 
shortfall  scores,  density  scores  can  get  bad  and  then  get  better  again  as 
new  words  are  added  to  a theory.  Hence  the  density  score  is  certainly  not 
guaranteed  to  be  an  upper  bound  of  the  expected  eventual  score.  However, 
it  has  another  interesting  property:  in  exactly  those  cases  where  it  does 

not  bound  the  eventual  score,  there  is  a potential  seed  word  for  the  same 
ultimate  theory  somewhere  else  in  the  utterance  that  has  a better  score 
density  and  whose  score  density  does  bound  the  eventual  score.  This  arises 
from  the  property  of  densities  that  the  density  of  two  regions  combined 
will  lie  between  the  densities  that  they  each  have.  It  turns  out  that  this 
alone  is  not  sufficient  to  guarantee  completeness  for  a density  scoring 
strategy,  since  it  is  still  possible  for  the  density  score  starting  from 
the  best  correct  seed  to  fall  below  that  of  some  other  less-than-optimal 
spanning  theory  before  it  can  be  extended  to  a complete  theory  itself. 
However,  with  the  addition  of  a facility  for  combining  islands  that  start 
from  separate  seeds  when  they  collide  with  each  other,  the  density  scoring 
strategy  working  middle-out  from  multiple  seeds  can  be  shown  to  be 
complete.  Again,  density  scoring  does  not  depend  on  any  assumptions  about 
the  basic  scores  to  which  it  is  being  applied  other  than  that  they  be 
additive  (and  capable  of  division).  Hence  the  density  method  can  be 
applied  to  either  the  original  quality  score  or  to  a shortfall  score.  The 
combination  of  the  two  methods  in  a shortfall  density  strategy  seems  to  be 
more  effective  than  shortfall  or  density  scoring  alone. 

7 • Ghosts 

In  the  middle-out,  island-driven  strategies,  each  time  an  event  is 
given  to  the  Syntactic  component  for  evaluation,  proposals  are  returned  and 
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word  matches  are  found  on  both  sides  (except  when  the  island  is  already  at- 
one ena  of  the  utterance) . Although  words  are  added  to  only  one  end  of  an 
island  at  a time,  any  theory  that  results  from  a given  event  will  make 
proposals  at  the  other  end  that  arc  either  the  same  as  or  a subset  of  the 
proposals  that  were  made  by  its  parent  theory.  Hence,  it  will  eventually 
have  to  use  a word  at  its  other  end  that  scores  no  better  than  the  best 
word  that  was  found  there  at  the  time  the  event  was  noticed. 

The  HWIM  system,  under  the  control  of  a flag,  can  keep  track  of  the 
words  that  were  found  at  the  other  end  of  an  island  when  an  event  is 
created.  We  refer  to  these  words  as  the  ghosts  for  the  event.  By  computing 
the  score  of  an  event  using  the  best  of  the  ghosts  as  well  as  the  words  of 
the  event  itself,  we  can  gain  an  effective  lookahead  of  one  word  in  the 
opposite  direction.  This  promotes  the  acceleration  of  good  events  up  the 
queue  and  bad  events  downwards.  We  also  remembar  the  proposals  used  to 
find  the  ghosts  and  if  the  extension  of  the  event  does  not  in  fact  result 
in  tightening  those  proposals,  then  the  ghost  words  can  be  used  again  for 
the  new  theory  without  having  to  re-call  the  Lexical  Retrieval  component. 

8.  Choosing  Direction 

As  mentioned  above,  whenever  an  event  is  processed,  new  words  are 

generally  found  at  both  ends  of  the  new  island.  Any  spanning  theory  that 

can  eventually  be  derived  from  such  an  island  can  be  reached  either  by 

adding  the  left  word  first  and  continuing  to  process  the  resulting  theory, 

or  by  adding  the  right  word  first  and  proceeding  from  there.  This  results 

in  a potential  combinatoric  explosion,  since  in  general  a theory  of  k words 

k 1 

can  be  derived  in  2 different  ways.  This  explosion  can  be  limited, 

however,  by  testing  every  event  that  is  noticed  to  see  if  the  theory  it 
produces  has  been  found  before.  The  problem  could  be  eliminated  completely 
by  choosing  arbitrarily  to  use  only  the  left  notices  or  the  right  notices 
(the  choice  can  be  made  to  depend  on  aspects  of  the  particular  situation 
such  as  the  direction  having  the  best  scoring  new  word  or  the  shortest 
distance  to  the  end  of  the  sentence)  . While  this  option  .is  available  in 
the  system  under  control  of  a flag,  there  are  advantages  to  pursuing  both 
directions  when  the  Lexical  Retrieval  component  is  given  a limit  on  the 
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number  of  words  that  it  may  retrieve.  In  this  situation,  it  is  possible 
that  extending  an  island  in  a given  direction  can  result  in  tighter 

predictions  at  the  other  end  and  may  make  it  possible  to  find  a word  there 
that  was  buried  in  the  noise  before.  The  CHOOSEDIR  option  gains  this 
benefit  while  effectively  controlling  the  combinatorics. 

Under  the  control  of  a flag  called  CHOOSEDIR,  HWIM  will  check  each 
time  an  event  is  processed  to  see  if  it  is  the  first  successful  event 

resulting  from  its  parent.  If  so,  HWIM  marks  that  direction  as  the  chosen 
one  for  all  events  stemming  from  that  parent,  and  rescores  all  related 
events  going  in  the  opposite  direction  using  the  worst  of  the  ghost  words 
instead  of  the  best.  The  effect  of  this  rescoring  is  to  demote  all  of  the 
events  going  opposite  to  the  chosen  direction  by  a score  equivalent  to  the 
difference  between  the  scores  of  the  best  and  worst  ghosts.  If  the 

proposals  used  in  retrieving  those  ghosts  are  relatively  lax,  or  the  score 
of  the  worst  ghost  is  sufficiently  high  that  calling  the  Retrieval 
component  back  with  tighter  predictions  might  result  in  new  words  with 
reasonable  scores,  then  this  demotion  is  not  severe.  Otherwise  it 

effectively  rules  these  events  out  of  consideration.  The  effect  is  that 
"wrong  way"  events  are  considered  only  if  they  have  a chance  of  producing 
new  words  with  a reasonable  score.  Moreover,  they  are  ranked  in  the  event 
queue  at  the  point  where  they  will  be  processed  only  when  a word  with  such 
a score  would  be  the  best  thing  to  do.  All  of  these  wrong  way  events  are 
blocked  from  noticing  any  of  the  words  in  their  ghost  list,  since  all  those 
words  have  already  been  pursued  in  the  chosen  direction.  This  feature 
gains  almost  all  of  the  combinatoric  control  that  choosing  only  one 
direction  would  gain,  while  giving  the  potential  for  using  tighter 
predictions  to  find  new  words  with  exactly  the  right  priority  ranking  ir. 
the  event  queue.  The  use  of  ghosts  and  the  chosen  direction  heuristics  do 
not  affect  the  completeness  of  strategies  that  would  otherwise  be  complete. 

9.  Other  Components  of  Score 

In  addition  to  the  basic  lexical  score  described  above  and  the  various 
priority  scores  that  can  be  derived  from  it,  there  are  three  other 
components  of  an  event  score.  These  are  an  ending  score  assigned  by  the 
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APR  component  to  boundaries  where  an  utterance  could  begir  o.'  end,  a 
verification  score . and  a syntactic  score ■ Ending  scores  are  zero  except 
when  a theory  hypothesizes  an  end  at  a boundary  that  the  APR  has  specified 
as  possible  but  unlikely.  Verification  scores  are  assigned  by  the 
Verification  component  based  on  a parametric  matching  between  hypothesized 
words  and  the  input  waveform.  They  are  calibrated  statistically  as 
described  in  Vol.  I.,  Sec.  B and  Vol.  II,  Sec.  D to  produce  log  likelihood 
ratios  that  are  compatible  with  the  Lexical  Retrieval  scores.  Syntactic 
scores  reflect  penalties  for  linguistic  inconsistencies  that  decrease  the 
likelihood  of  an  interpretation  but  do  not  rule  it  out  altogether;  they  are 
also  in  the  form  of  log  likelihood  ratios.  In  all  of  the  experiments  that 
we  have  so  far  run,  the  syntactic  scores  have  always  been  zero.  Mechanisms 
have  been  set  up  to  compare  prosodic  cues  in  ♦'he  signal  against  prosodic 
expectations  in  the  grammar  and  return  the  result  as  a component  of  the 
syntactic  score  (Vol.  IV,  Sec.  C),  but  we  have  not  been  able  to  run 
experiments  to  test  this  facility.  All  three  of  these  "non-lexical"  scores 
are  converted  to  score  densities  when  one  of  the  density  scoring  strategies 
is  being  used.  The  syntactic  and  verification  scores  (like  the  lexical 
match  score)  are  divided  by  the  duration  of  the  island;  the  ending  score  is 
divided  by  the  duration  of  the  utterance.  The  use  of  verification  scores 
and  syntactic  scores  can  affect  the  theoretical  completeness  of  a strategy 
that  is  otherwise  complete. 

10.  Control  Strategy  Options 

The  selection  of  a particular  strategy  in  HWIM  is  determined  by  the 
settings  of  global  flags  that  govern  various  strategy  options.  In  the  final 
version  of  the  system  there  are  25  such  flags,  with  a total  number  of 
permissible  combinations  exceeding  5000.  In  this  section  we  will  describe 
the  major  such  flags.  Other  flags  and  other  control  variables  govern  such 
things  as  a limit  on  the  number  of  events  to  be  processed  before  giving  up, 
stack  lengths  for  various  requests  to  Lexical  Retrieval,  and  a variety  of 
experimental  control  strategies. 
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Major  Control  Flags : 


a)  PRONLIKEFLAG 

The  phonlogical  rule  component  includes  a facility  for  taking 
pronunciation  likelihood  estimates  from  the  base-form  pronunciations  of 
words  and  combining  them  with  likelihood  multipliers  indicated  in  the 
optional  phonological  rules  [Woods  et  al.,  1975]  to  produce  likelihood 
estimates  for  each  pronunciation.  The  Lexical  Retrieval  component  returns 
these  likelihoods,  expressed  as  log  likelihood  ratios,  with  each  word  match 
that  it  finds,  but  keeps  this  number  separate  from  the  lexical  score  so 
that  the  Control  component  can  decide  whether  to  use  it  or  not.  When 
PRONLIKEFLAG  is  T,  Control  uses  the  lexical  score  plus  the  pronunciation 
likelihood  as  the  quality  score  for  the  word.  When  PRONLIKEFLAG  is  NIL, 
only  the  lexical  score  is  used.  In  all  of  the  experiments  described  in 
this  report  (Sec.  F;  Vrl.  I,  Sec.  D),  PRONLIKEFLAG  is  T. 

b)  SHORTFALLFLAG 

As  discussed  above,  one  of  the  strategies  involves  using  a shortfall 
score  instead  of  the  straight  quality  score  for  ordering  the  queue  of 
events  to  be  done.  When  SHORTFALLFLAG  is  T.  a MAXSEG  profile  is  constructed 
and  the  priority  score  of  an  event  is  computed  using  the  sum  of  the 
differences  between  tie  the  actual  score  and  the  MAXSCORE  of  each  word 
match  involved. 


c)  DENSITYFLAG 


When  the  DENSITYFLAG  is  set,  durations  are  carried  with  every  event,  and 
event  scores  are  computed  using  either  the  quality  or  shortfall  scores 
(depending  on  SHORTFALLFLAG)  divided  by  this  duration.  Individual  word 
matches  in  fuzzy  word  matches  are  also  sorted  by  density  order  in  this 
case,  and  the  non-lexical  components  of  tne  score  (end  penalties, 
verification  scores,  and  syntactic  scores)  are  divided  by  durations  (as 
described  above)  to  make  them  comparable. 

d)  VERIFY FLAG 


VERIFYFLAG  has  four  possible  values:  NIL,  VERIFYgNOTICE,  VERIFY0DO , 
and  VERIF Y0PICK . The  first  calls  for  no  verification  at  all.  The  second 
indicates  that  the  Verifier  should  be  called  when  new  words  are  noticed,  so 
that  the  score  assigned  to  the  event  includa?  the  verification  score  of  the 
new  word.  VERIFYiDO,  on  the  other  hand,  indicates  that  verification  should 
wait  until  the  time  that  the  event  is  processed.  Thus  the  score  of  an 
event  in  the  e^ent  queue  does  not  include  tne  verification  score  of  the  new 
word,  but  does  so  for  all  of  the  words  in  the  parent  theory.  VERIFY^DO 
results  in  fewer  calls  to  the  Verification  component,  but  does  not  use  the 
verification  information  as  early  as  possible. 


VERIFY0PICK  provides  the  best  features  of  the  other  two  options.  It 
gives  each  event  a default  verification  credit  at  the  time  it  is  noticed, 
which  is  comparable  to  the  best  possible  verification  score  (the  amount  of 
score  to  default  is  specified  by  a control  variable).  Then,  at  the  time 
when  the  event  reaches  the  top  of  the  event  queue  and  is  about  to  be 
processed,  Verification  is  called  for  the  new  word  to  replace  the  default 
score  by  a real  one.  If  the  rescored  event  is  still  the  best  scoring 
event,  then  it  is  processed.  Otherwise  it  is  re-inserted  at  the 
appropt  iate  point  in  the  queue,  and  the  process  is  repeated  for  the  new  top 
item  In  the  queue.  The  system  can  tell  the  difference  between  an  event 
that  includes  a default  verification  score  and  one  that  has  been  verified, 
so  that  when  an  already  verified  event  reaches  the  top  of  the  queue  it  will 
not  be  verified  again.  VERIFY0PICK  thus  effectively  uses  the  verification 
scores  of  all  of  the  words  in  an  event  in  deciding  which  event  is  to  be 
processed  (gaining  the  advantage  of  the  VERIFY@NOTICE  strategy),  while 
actually  calling  the  Verirication  component  for  only  those  events  that  make 
it  to  tne  top  of  the  queue  (the  advantage  of  the  VERIFY0DO  option). 
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e)  HYBRIDFLAG 

HYBRIDFLAG  has  three  possible  values:  NIL,  0.  and  a non-zero  integer. 
When  HYBRIDFLAG  is  zero,  the  initial  scan  of  the  utterance  for  seed  words 
consists  only  of  words  beginning  at  boundaries  that  the  APR  component  has 
labeled  as  possible  left  ends.  When  it  is  an  integer  greater  than  zero, 
the  initial  scan  is  permitted  to  go  beyond  this  point  by  a number  of  frames 
(10  millisecond  intervals)  equal  to  the  value  of  HYBRIDFLAG.  In  either 
case,  when  HYBRIDFLAG  is  non-NIL,  theories  are  not  permitted  to  notice 
words  on  the  right  until  they  have  been  extended  to  the  left  end  of  the 
utterance  and  the  left  end  event  has  been  done.  Hybrid  strategies  permit 
skipping  the  first  word  or  two  of  the  utterance  in  order  to  find  a robust 
anchor  word,  but  do  so  for  at  most  a bounded  distance  into  the  utterance. 
They  shift  to  left-to-right  processing  as  soon  as  possible  and  appear  to 
have  a good  combination  of  the  advantages  of  both  left-to-right  and 
middie-out  strategies. 

f)  GHOSTFLAG 

GHOSTFLAG  causes  the  control  component  to  include  with  each  event  a 
record  of  the  words  that  were  noticed  at  the  other  end  of  the  theory  when 
the  event  was  created.  The  best  such  ghost  word  is  then  used  in  addition 
to  the  words  of  the  theory  and  the  new  word  to  determine  the  score  of  the 
event . 


g)  CHOOSEDIRFLAG 

When  CHOOSEDIRFLAG  is  T,  the  direction  choosing  strategy  [Sec.  E.8]  is 
enabled.  That  is,  when  the  first  successful  event  for  a given  parent 
theory  is  found,  all  events  spawned  by  that  parent  going  in  tne  opposite 
direction  are  rescored  using  their  worst  ghost  word  and  are  blocked  from 
noticing  any  of  their  ghost  words.  Their  only  function  is  to  try  to  find 
additional  words  at  the  chosen  end  by  making  more  specific  proposals 
resulting  from  larger  context.  CHOOSEDIHFLAG  can  only  be  enabled  when 
GHOSTFLAG  is  set. 

h)  COLLISIONFLAG 

COLLISIONFLAG  enables  the  island  collision  mechanism.  When  it  is  on, 
each  event  is  entered  into  one  of  two  island  tables  depending  on  the 
direction  of  the  event.  The  other  island  table  is  then  consulted  to 
determine  if  the  same  word  at  approximately  the  same  place  has  been  noticed 
in  the  other  direction.  If  so,  a collision  event  is  created,  which  when 
processed  will  combine  the  two  noticing  theories  and  the  new  word  between 
them  into  a single  theory.  This  event  is  scored  using  all  of  the  words  of 
the  two  theories  plus  the  new  word  (plus  the  gnosts  of  both  colliding 
events  if  GHOSTFLAG  is  on). 


1 1 . The  12  October  Strategies 

As  mentioned  earlier,  all  of  HWIM's  control  strategy  options  fail 
within  a general  island-driven  framework.  They  each  perform  an  initial 
scan  of  some  region  of  the  utterance,  creating  initial  one-word  seed 
events.  In  general  "middle-out"  strategies,  the  initial  scan  is  done  over 
the  entire  utterance.  In  "left-to-right""  strategies,  the  initial  scan 
only  considers  words  that  could  begin  the  utterance  (i.e.,  whose  left 
boundary  falls  at  one  of  several  possible  points  that  have  been  labeled  by 
the  APR  component  as  possible  starting  points  of  the  utterance) . In 
"hybrid"  strategies,  the  initial  scan  is  done  on  a fixed  initial  portion  of 
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the  utterance  (from  its  beginning  to  some  point  a specified  number  of 
milliseconds  later).  Then  a middle-out  analysis  is  done  on  this  region 
with  the  remainder  necessarily  being  analyzed  left-to-right . In  all  of 
these  strategy  options,  events  are  ordered  on  the  queue  by  their  priority 
scores.  The  choice  of  a particular  combination  of  strategy  options  is 
determined  by  the  settings  of  the  control  flags  described  above.  In  this 
section,  we  will  describe  some  particular  combinations  that  we  have  used 
for  testing  the  final  speech  understanding  system  as  of  October  1976. 

The  preferred  algorithm,  which  we  chose  on  the  basis  of  preliminary 
experiments  to  use  for  our  final  test  run,  is  the  shortfall  density 
strategy  running  in  hybrid  mode  with  an  initial  hybrid  search  displacement 
of  0.  (We  refer  to  such  a hybrid  strategy  as  "left  hybrid".)  The  strategy 
uses  the  Verification  component  with  the  VERIFYgPICK  option,  and  includes 
the  use  of  ghosts  although  they  don't  contribute  anything  except  for  a few 
seeds  not  immediately  at  the  left  end.  As  a convenient  mnemonic  for  naming 
this  and  other  strategies,  we  use  the  abbreviations  LH  for  left  hybrid,  SD 
for  shortfall  density,  and  VP  for  verify  at  pick.  The  strategy  just 
described  is  thus  refered  to  a LHSDVP.  The  corresponding  middle-out 
strategy,  not  using  the  hybrid  option  would  be  simply  SDVP.  Alternatives 
to  SD  are  S for  shortfall  alone,  Q for  quality  alone,  and  QD  for  quality 
density.  Alternatives  for  VP  are  VD  for  verify  at  do,  VN  for  verify  at 
notice,  and  NV  for  no  verify. 

The  strategies  compared  in  experiments  with  the  final  system  include: 

a)  LHxNV  for  x = S,  SD,  Q.  QD 

i.e.,  Left  hybrid  strategy  without  Verification,  using  the 
different  priority  scoring  metrics — S,  SD,  Q and  QD. 

b)  xNV  for  x = S,  SD,  Q,  QD 

i.e.,  Middle  out  strategy  without  Verification  using  different 
priority  scoring  metrics.  Ghosts,  island  collisions,  and 
direction  choosing  heuristics  are  all  used  in  these  experiments. 

c)  LHSDx  for  x = VP,  VN , VD,  NV 

i.e.,  Left  hybrid  shortfall  density  using  various  Verification 
options . 

d)  SD+x  for  x = 0,  C,  G,  GD,  GDC 

i.e..  shortfall  density  without  Verification  using  different 
combinations  of  the  ghosts,  island  collision,  and  direction 
choosing  heuristics.  SD+0  uses  none  of  the  three  options,  C 

choosing f? CHOOSEDIR ) ?01 1 ^ sions ' G for  ghosts,  and  D for  direction 
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F.  SYSTEM  PERFORMANCE 


The  performance  of  the  latest  (12  October)  version  of  HWIM  was 

demonstrated  by  running  it  with  vocabularies  of  two  sizes  on  a test  set  of 

124  3entence-lcngth  utterances,  as  described  in  Vol.  I,  Sec.  D.  To 

recapitulate,  these  results  are  shown  in  Table  1 below. 

BIGDICT  MIDDICT 

( 1 097  words)  (409  words) 


Correctly  understood 

54 

44% 

65 

= 52? 

All  words  correct 

51 

= 

41? 

59 

= 48? 

Semantically  correct 

3 

Z 

2? 

o 

= 4? 

Incorrect 

45 

- 

36? 

40 

= 32? 

Close  to  correct 

29 

= 

23? 

25 

= 20? 

Less  correct 

13 

z 

10? 

10 

= 8? 

Very  wrong 

3 

z 

2? 

5 

= 4? 

No  response 

2$ 

- 

20? 

15 

= 15? 

Gave  up  (150  theories) 

24 

z 

19? 

18 

= 14? 

System  broke 

1 

z 

1? 

1 

= 1? 

Estimated  average 

branching  (words) 

196 

67 

Speed  (times  real 

time) 

1350 

1050 

Table  1.  Summary  of  final  performance  results, 
124  utterances  by  three  speakers. 


The  test  sentences  are  listed  in  Vol.  1,  Appendix  1,  and  the  performance 
results  for  each  test  utterance  are  listed  ir:  Appendix  3,  this  volume. 
This  section  discusses  additional  aspects  of  system  performance  - speed, 
speaker  variation,  and  the  results  of  some  experiments  in  strategy 
variation  performed  with  a small  subset  of  the  124  utterances. 

1 . Real  Time  Issues 

One  of  the  objectives  of  the  ARPA  Speech  Understanding  research 
program  was  to  discover  and  develop  te-hniques  that  would  permit  the 
understanding  of  large-vocabulary,  continuous  speech  in  a few  times  real 
time  on  a 100  mips  (millions  of  instructions  per  second)  machine.  This 
target  was  established  on  the  grounds  that  the  machines  on  which  speech 
understanding  systems  would  eventually  run  would  be  much  faster  than  the 
machines  available  at  the  initiation  of  the  project  in  1971.  By  way  of 
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comparison,  the  DEC  PDF-10  KA  processor  on  which  the  BBN  speech 
understanding  work  has  been  done  is  approximately  a .3*1  mips  machine 
[Fuller,  1976].  This  is  a relatively  slow  machine  by  today's  standards, 
but  one  on  which  a great  deal  of  sophisticated  software  has  been  developed. 

For  a research  project  such  as  this  one,  the  prior  existence  of  the 
software  necessary  for  experimental  system  development  is  far  more 
important  than  basic  machine  speed,  although  other  factors  being  equal,  a 
faster  machine  is  clearly  preferable.  In  our  case,  the  existence  of  an 
interactive  INTERLISP  system  and  the  flexibility  of  the  TENEX  file  and 
process  structure  — especially  the  multiple  process  capabilities  — have 
contributed  significantly  to  the  project.  The  development  of  such 
facilities  to  take  maximum  advantage  of  a new  machine  may  require  several 
years.  Since  the  five  year  time  period  set  out  for  the  speech 
understanding  program  was  considered  barely  enough  time  to  do  the  necessary 
experimental  development  of  the  speech  systems  themselves,  it  was  not 
feasible  to  work  in  the  context  of  untested  machines  and  operating  systems. 
This  decision  meant  that  the  experimental  systems  that  we  developed  would 
be  considerably  slower  than  would  be  possible  by  running  on  appropriate 
hardware  configurations. 

In  addition  to  this  basic  factor  of  machine  speed,  we  elected  to 
accept  an  additional  inflation  in  the  running  time  of  our  own  system  in 
order  to  gain  the  flexibility  for  experimentation  and  system 
reconfigurability  that  comes  from  using  high-level  programming  languages 
such  as  INTERLISP  and  from  using  styles  of  programming  that  promote 
changeability  at  the  cost  of  run-time  efficiency.  For  example,  INTERLISP 
spends  extra  instructions  checking  data  types  and  array  bounds,  a procedure 
that  greatly  reduces  the  debugging  cost.  In  addition,  extra  instructions 
are  spent  1)  in  keeping  records  of  the  current  state  of  a running  program 
so  that  they  will  be  available  for  inspection  by  a programmer  at  any  point 
in  the  process;  and  2)  in  testing  flags  to  determine  which  options  to  use 
at  various  points  in  the  programs,  so  that  experimental  comparisons  of 
different  strategies  can  be  easily  carried  out.  Trace  instructions  were 
also  embedded  at  significant  points  in  the  code  so  that  comprehensive 
records  of  the  intermediate  stages  of  the  understanding  process  could  be 
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kept  and  studied  (an  example  of  such  a trace  is  shown  in  Vol.  I,  Appendix 

2). 


Given  that  today's  experimental  speech  understanding  systems  run 
significantly  slower  than  ones  that  would  be  constructed  for  applications, 
a question  of  interest  is  what  speeds  would  be  possible  if  the  techniques 
used  in  the  BBN  speech  understanding  system  were  appropriately  implemented 
on  suitable  hardware? 

We  will  assume  that  the  signal  processing,  which  requires 
approximately  55  times  real  time  on  our  current  facility,  can  be  done  in 
real  time  either  with  analog  circuitry,  or  with  fast  digital  processors. 
Similar  kinds  of  processing  are  already  being  done  in  real  time  on  such 
processors  as  the  SPS-41  and  the  Lincoln  Laboratory  DVT  [Hofstetter  et  al . , 
1975].  The  remainder  of  the  system  runs  in  approximately  1300  times  real 
time,  that  is,  the  number  of  seconds  of  computer  time  required  to 
understand  an  utterance  is  approximately  1300  times  the  duration  of  the 
utterance  in  seconds  Some  utterances  can  be  understood  in  less  time,  and 
some  require  longer,  but  this  is  an  approximate  average. 

Of  the  total  time  required  by  the  non-signal  processing  parts  of  the 
system,  the  Acoustic-Phonetic  Recognition  (APR)  component  takes  a 
negligible  amount  of  time,  and  the  remainder  is  broken  down  roughly  as 
shown  below.  (Here  "ss"  stands  for  "second  of  speech.") 

Component  Percentage  CPU  sec  ./ss 


Lexical  retrieval 

40? 

520 

Verification 

15? 

195 

Control  and  Syntax 

40? 

520 

Tracing 

5? 

65 

(The  figure  given  for  tracing  time  is  that  of  a single  function 
PRINTEVENTQ,  which  prints  the  event  queue,  the  most  time-consuming  single 
tracing  function.  There  are  many  other  tracing  functions  embedded 
throughout  the  Control  and  Syntax  components,  so  the  actual  trace  time  is 
somewhat  higher  than  indicated  and  the  Control  and  Syntax  time  is  therefore 
somewhat  less  than  indicated.  Of  course,  in  a non-ex  per imental  system, 
this  tracing  would  not  be  done.) 
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One  of  the  overheads  of  jobs  running  on  TENEX  is  due  to  the  demand 
paging  access  to  a virtual  memory  that  TENEX  provides.  Each  running 
process  (called  a "fork")  has  a virtual  memory  of  262,144  words,  which  is 
mapped  into  a working  set  of  in-core  pages,  each  consisting  of  512  words. 
Whenever  a running  process  accesses  a memory  location  that  is  not  in  core, 
the  process  waits  while  the  operating  system  fetches  the  necessary  page 
into  core  memory,  possibly  removing  another  page  from  the  working  set  to 
make  room.  This  procedure  is  called  a "page  fault"  and  the  operating 
system  charges  the  process  for  running  the  "pager  trap  code"  that 
determines  how  to  swap  the  required  page  into  core.  This  time  is  included 
in  all  of  the  system's  clock  times  and  is  impossible  to  separate  out  from 
the  time  required  for  actual  instructions  in  the  running  process.  When  a 
system  is  lightly  loaded  and  an  individual  process  is  permitted  to 
accumulate  a large  working  set  in  core,  the  number  of  page  faults  can  be 
fev,  but  when  the  load  is  heavy  and  the  process  is  swapped  out  frequently, 
a process  can  spend  significant  amounts  of  page  fault  time  retrieving  the 
pages  it  has  lost.  To  make  matters  worse,  when  one  fork  calls  another,  the 
pages  of  the  idle  calling  fork  are  made  available  for  reuse  at  the  same 
time  that  the  called  fork  is  demanding  pages,  so  that  there  is  likely  to  be 
significant  page  contention  between  two  "cooperating"  forks.  Since  the 
speech  understanding  system  contains  a number  of  forks  that  call  each 
other,  this  effect  is  significant. 

The  upshot  of  the  above  discussion  is  that  there  is  a largely 
uncontrollable,  variable,  and  not  adequately  measurable  component  due  to 
page  faulting  in  all  of  the  times  that  we  have  measured  for  our  system.  If 
the  system  were  running  in  an  environment  where  the  necessary  number  of 
pages  of  real  memory  were  available  to  the  processor  at  all  times,  all  of 
this  time  would  go  away.  Average  TENEX  system  performance  shows  that  about 
12?  of  the  "sold"  time  (time  charged  to  user  processes)  is  spent  handling 
pager  traps.  The  HWIM  system,  with  its  multiple  INTERLISP  forks  and  large 
data  structures,  can  be  expected  to  incur  page  faults  more  often  than  the 
average;  it  seems  safe  to  estimate  that  something  like  15$  of  the  CPU  time 
measured  by  the  system  is  page  faulting  time  (and  the  actual  amount  may  be 
significantly  more) . 
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Major  improvements  in  time  requirements  can  be  gained  by  reproerammirig 
several  of  the  components  for  speed.  This  has  been  done  to  some  extent 
already  with  the  Lexical  Retrieval  component,  which  has  been  written  in 
BCPL,  a language  that  compiles  into  fairly  efficient  machine  code. 
However,  the  Lexical  Retrieval  component  currently  includes  overhead  for  a 
variety  of  options  we  are  not  now  using,  and  writing  in  BCPL  does  not  give 
one  as  efficient  access  to  tie  accumulators  of  the  machine  as  would  machine 
language.  We  estimate  a factor  of  two  could  be  gained  from  eliminating 
unused  parts  of  the  code  and  by  recoding  from  BCFL  into  machine  language. 

The  Control  and  Syntax  components  of  the  system  are  both  implemented 
in  INTERLISP,  a language  that  gives  great  flexibility  in  experimental 
program  development,  while  charging  a fairly  high  penalty  in  overhead.  For 
example,  in  the  timing  breakdown  printed  on  the  traces,  a very  low  level 
function,  SHCRTORDERP,  was  found  to  take  about  4 milliseconds  per  call. 
All  this  function  does  is  test  a global  flag  to  determine  which  of  two 
comparisons  to  make  and  then  compare  two  floating  point  numbers  accessed 
from  an  array  to  determine  which  is  greater.  It  is  almost  an 
understatement  to  s-y  that  the  effect  of  this  function  can  easily  be  done 
in  less  than  100  microseconds.  (The  rest  of  the  time  is  spent  in  creating 
INTERLISP  stack  entries,  adding  the  names  of  the  variables  so  that  they 
will  be  available  for  subsequent  stack  scans,  checking  that  the 
subfurctions  called  are  indeed  defined  functions  and  that  the  arrays 
accessed  are  defined  arrays  and  the  indices  are  within  bounds,  etc.)  This 
would  indicate  that  a factcr  of  at  least  40  in  speed  could  be  gained  in 
reprogramming  the  INTERLISP  portions  of  the  system  into  machine  language. 
Of  course,  this  is  just  one  function,  but  it  is  executed  in  one  of  the 
innermost  loops  of  the  system.  Previous  experience  with  translating 
INTEPLISP  code  into  machine  language  indicates  that  speedups  of  this 
magnitud  are  indeed  achievable.  Speedups  of  a factor  of  10  have  been 
achieved  even  within  INTERLISP  by  block  compiling  groups  of  functions 
together,  and  this  has  not  yet  been  done  to  the  code  in  question. 

The  CPU  times  measured  for  the  Control  and  Syntax  components  also 
include  the  setup  times  for  the  Syntax  and  TRIP  forks.  These  setup  times 
should  not  have  been  included  in  the  run  time  measurements,  but  were  done 
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so  automatically,  given  the  way  we  had  implemented  the  functions.  Hence, 
the  measured  times  for  the  Control  componert  are  high  by  this  amount 
(approximately  90  seconds  of  CPU  time  per  utterance,  regardless  of 
duration).  Taking  everything  into  account,  we  feel  safe  in  projecting  a 
factor  of  40  speedup  in  the  Control  and  Syntax  components  by  reprogramming 
to  machine  language. 

The  Verification  component  is  already  implemented  in  a combination  of 
Fortran,  BCPL,  and  machine  language.  Further  gains  in  its  efficiency  are 
expected  to  be  small. 

Summarizing  the  above  discussion,  we  would  project  the  following  times 
to  be  realizable  on  our  current  computer  configuration  by  reprogramming 
individual  components  for  speed: 

Component  Current  CPU/ss  Projected  CPU/ss 


Lexical  retrieval 

520 

260 

Verification 

195 

195 

Control  & syntax 

520 

13 

Tracing 

65 

0 

1300 

~468 

If  this  were  implemented  on  a dedicated  FDP-10  with  sufficient  core  memory 
to  eliminate  page  faulting  (or  make  it  insignificant),  an  additional 
improvement  of  at  least  19$  would  result,  bribing  the  total  time 
requirements  to  400  CPU  seconds  per  second  of  speech  (on  a .34  mips 
machine) . 

On  a machine  with  the  capability  of  100  mips  (an  instruction  time  of 
10  nanoseconds),  this  would  require  only  1.36  CPU  seconds  per  second  of 
speech  (or  1.36  times  real  time),  thus  exceeding  the  projected  goals  of  "a 
few  times  real  time". 

All  of  the  above  discussion  assumes  a single  serial  processor  with  the 
speeds  indicated.  However,  the  speech  understanding  algorithms  developed 
appear  to  lend  themselves  well  to  parallel  implementation.  At  the  control 
level,  the  task  of  understanding  a sentence  consists  of  a search  in  a space 
whose  depth  is  the  number  of  words  in  the  utterance,  but  whose  branching  is 
a function  of  the  competition  from  oth *r  partial  theories  that  are 
acoustically  similar.  We  typically  process  on  the  order  of  100  ev  nts  to 
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determine  the  chosen  interpretation,  but  only  on  the  order  of  10  of  them 
( i • e . , the  number  of  words  in  the  utterance  plus  two  end  events)  are  actual 
teps  in  the  construction  of  the  chosen  theory.  The  others  are  competitors 
that  were  evaluat'd  to  determine  which  theory  was  best.  Thus  it  looks  like 
almost  a factor  of  10  in  para’.iism  might  be  achieved  for  our  current 
system,  by  using  a configuration  of  10  p*  cessors  sharing  a common  memory. 
CMU  has  been  exp’oring  such  pvrallel-processor  implementations  of  speech 
understanding  systems.  With  such  configurations,  the  necessary  1 36  mips 
for  real  time  operation  could  be  achieved  using  processors  with  75 

nanosecond  instruction  times. 

All  <-f  the  above  discussion  assumes  no  bas.  : change  in  the  algorithms 
or  performance  of  the  current  system . Undoubtedly  there  are  improvements 
that  remain  to  be  discovered  in  the  algorithms  that  would  yield  further 
speed  improvement.  Moreover,  a major  factor  not  considered  in  the  above 
analysis  is  the  effect  of  segmentation  errors  and  gross  mislabelings  by  the 
APR  component  on  the  overall  number  of  events  that  must  be  processed  to 
determine  the  chosen  interpretation.  Current  experience  indicates  that, 
due  to  a cascade  effect  through  the  various  processing  levels,  an  error  in 
segmenting  or  labeling  at  the  APR  level  can  give  rise  to  a number  of  extra 
events  being  processed  at  the  control  level  before  the  correct 

interpretation  emerges  (if  it  does  so  before  an  incorrect  interpretation  is 
discovered).  The  number  of  such  events  depends  on  the  severity  of  the 
original  error  in  a non-linear  way  so  that  doubling  the  severity  of  an 
error  more  than  doubles  the  number  of  extra  events  to  be  processed. 
Consequently,  a small  decrease  in  the  number  and  severity  of  APR  errors  can 
result  in  a significant  decrease  in  the  number  of  events  processed.  Thus, 
an  improvement  in  APR  performance  stati’  tics  alone  cat!  significantly 
improve  the  overall  speed  of  the  system  inde;  ndent  of  any  implementation 
efficiency  improvements.  We  can  therefore  expect  that  continued  extension 
of  the  range  of  acoustic-phonetic  events  that  the  <;PR  component  recognizes 
and  handles  correctly  would  result  in  an  additional  speedup  of  the  system. 
A factor  of  two  here  would  not  be  unreasonable  to  expect. 
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Post  Issues 

Another  relevant  question,  especially  given  the  previous  discussion  of 
machine  speeds  and  multiprocessor  configurations,  is  the  expected  cost  of 
such  systems.  One  might  expect  that  such  a machine  would  be  a giant 
configuration  costing  millions  of  dollars.  However,  the  costs  of  computer 
processors  and  memory  are  currently  coming  down  dramatically  in  conjunction 
with  speed  improvements.  In  a study  done  at  the  Harvard  Business  School, 
John  Doerr  [1976]  estimates  that  by  1980  a 10  MIPS  system  suitable  for  a 
speech  understanding  system  could  be  built  for  about  $25,000.  His  system 
consists  of  a $500  CPU,  $3,600  fast  primary  memory,  and  $22,500  secondary 
memory.  At  this  rate,  a 10  processor  configuration  sharing  the  secondary 
memory  would  cost  only  $63,500  and  would  have  an  effective  capacity  of  100 
mips,  for  the  BBN  speech  understanding  task. 

2.  Performance  vs.  Speaker 

The  system  performance  for  each  speaker  is  shown  in  Table  2 below. 


BIGDICT 

TRAVELDICT 

WAV.' 

JJW 

MS 

WAW 

m 

RMS 

Correct 

30 

13 

1 1 

36 

16 

13 

Incorrect 

20 

q 

16 

20 

7 

13 

No  response 

12 

9 

4 

6 

8 

5 

Table  2.  Number  of  sentences  in  3 response  categories, 
broken  down  by  speaker. 

Although  there  are  differences  across  speakers,  a chi-square  test  shows 
that  the  differences  in  each  case  are  significant  only  at  a level  of  about 
p=0.25.  That  is,  there  is  no  strong  evidence  that  the  observed  differences 
across  speakers  did  not  occur  by  chance. 

3-  Accuracy  vs.  Sentence  Length 

The  "correctly  understood"  performance  (of  the  BIGDICT  system)  vs. 
utterance  length  in  words  and  duration  in  seconds  is  shown  in  Tables  3 and 
4 below: 
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Words  Accuracy 


Duration  ( sec)  Accuracy 


3 

1/8  = .12 

0.7  - 

1.0 

1/6  = 

4 

6/14  = .43 

1.0  • 

1.2 

7/10  = 

5 

18/30  = .60 

1.4 

6/14  = 

5 

13/24  = .54 

1.6 

6/15  = 

7 

8/20  = .40 

1.8 

12/22 

6 

6/18  = .33 

2.0 

8/17  = 

9 

0/2  = 0. 

2.2 

5/15  = 

10 

2/6  = .33 

2.4 

4/7  = 

13 

0/2  = 0. 

2.6 

3/7  = 

2.8 

2/6  = 

2.8  - 

4.0 

0/5  = 

Table 

3.  Accuracy  vs. 

Table 

4.  Accurac; 

v vs . 

utterance  length.  utterance  duration. 


.17 

.70 


.47 

• 33 

:8 
• 33 
0. 


(In  Table  3.  "words”  are  counted  as  lexical  entries  as  shown  in  Appendix  5. 
Besides  standard  English  words,  there  are  a few  compound  adjectives 
(ONE-WAY,  ROUND-TRIP)  and  place  names  (EL0PASO),  and  the  plural  morpheme  is 
a separate  word  (Bill's  = BILL  -S).) 


Table  3 shows  that  the  test  set  contained  relatively  few  utterances 
shorter  than  4 or  longer  than  8 words  long.  The  other  data  points  are  not 
sufficient  to  allow  one  to  make  any  quantitative  conclusions  about  the 
nature  of  the  decline  in  performance  as  the  length  of  the  sentence  grows. 
The  3-  and  4-word  data  points  are  of  some  surprise,  since  one  might  expect 
there  to  be  fewer  chances  for  the  system  to  go  wrong  with  such  short 
sentences . 


It  would  seem  that  the  3-word  sentences,  and  possibly  the  4-word  ones 
as  well,  may  be  spoken  differently  than  longer  ones.  Sentences  like  "Print 
her  fare”  and  "Figure  his  expenses”  seem  to  lack  the  flow  of  the  longer 
ones  like  "Schedule  a trip  by  train  to  New  York."  They  sound  more  like  a 
series  of  isr.ated  words,  and  this  may  affect  their  acoustic-phonetic 
characteristics . 


Furthermore,  there  are  "ending  effects"  that  should  be  relatively  more 
important  for  a short  sentence  than  for  a long  one.  For  example,  the  ends 
of  the  utterance,  where  the  speech  begins  with  maximum  subglottal  pressure, 
and  also  where  the  speech  ends,  are  '■’cgicns  of  rather  different 
acoustic-phonetic  characteristics.  Also,  in  a short  sentence,  there  is 
ss  syntactic  context  and  hence  less  syntactic  constraint  at  each  choice 
point.  Such  effects  may  swamp  the  high  expectations  of  a monotonic 
performance  vs.  length  model. 
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Much  the  same  can  be  said  about  the  performance  vs.  duration  data.  A 
rather  larger  set  of  test  utterances  than  we  had  the  resources  to  process 
would  have  been  necessary  to  explore  these  questions  more  fully. 

4 . Fxper iments  in  Strategy  Variations 

The  "LHSDVP"  control  strategy  used  in  the  final  (12  October)  version 
of  HWIM  for  the  performance  tests  described  above  represents  only  one  of  a 
large  number  of  strategy  variations  that  are  controlled  by  the  values  of 
about  25  flag  variables  in  the  Control  component.  The  selection  of  this 
particular  strategy  was  based  on  experimentation  on  the  effects  of  the 
various  heuristics  on  system  performance  during  the  development  of  the 
Control  component.  At  the  end  of  the  project,  it  would  have  been 
instructive  to  have  tested  many  strategy  variations  on  a large  test  set  of 
utterances  in  order  to  evaluate  their  effects  quantitatively,  as  has  been 
done  by  Paxton  [1Q76].  However,  time  limitations  prevented  us  from  doing 
that  with  a test  set  as  large  as  124  utterances. 

As  a compromise,  a set  of  10  utterances  was  selected  randomly  from  the 
set  of  124.  Four  strategy-variation  experiments  were  performed  by  running 
that  smaller  set  of  utterances  on  each  of  14  versions  of  HWIM.  Such  a 
small  test  set  cannot  give  significant  quantitative  results,  but  it  is 
hoped  that  it  will  give  the  "flavor"  of  the  heuristics  involved.  The 
per-utterance  results  from  these  four  experiments  are  presented  in  Appendix 
4 of  this  volume.  The  remainder  of  this  section  briefly  discusses  those 
results . 

Experiment  1:  LHxNV 

This  experiment  used  the  same  left-hybrid  strategy  as  the  12  October 
final  version  of  HWIM,  but  without  Verification;  the  experimental  variable, 
x,  was  the  method  of  scoring  hypotheses  - either  quality  (Q  i.e.,  LHQNV) , 
quality  density  ( QD  - LHQDNV),  shortfall  (S  - LHSNV),  or  shortfall  density 
(SD  - LHSDNV).  In  all  four  of  these  strategy  variation  experiments,  the 
"give  up"  threshold  was  set  to  100  theories  rather  than  150. 
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As  can  be  seen  in  Appendix  4,  two  utterances  yield  no  useful 
comparisons,  because  for  all  four  methods,  processing  was  terminated  before 
a spanning  theory  was  formed.  We  can  summarize  the  results  as  shown  below. 
(The  row  labeled  NTH5  gives  the  total  number  of  theories  required  for  the 
five  utterances  understood  correctly  by  methods  Q,  QD,  and  SD.) 


Correct 

LHQNV 

6 

LH£DNV 

5 

LHSNV 

0 

LHSDNV 

6 

Incorrect 

1 

1 

0 

2 

No  response 

, 3 

4 

10 

2 

NTH5 

1 145 

268 

- 

183 

Shortfall  scoring  (S)  is  so  breadth-first  that  no  sentence  was 
understood  in  as  few  as  100  theories.  Quality  density  (QD)  is  also 
significantly  more  breadth-first  than  the  remaining  two,  as  shown  by  its 
value  of  NTH5  and  the  larger  number  of  utterances  it  failed  to  finisn. 
Quality  and  shortfall  density  get  the  same  number  correct,  with  quality 
requiring  fewer  theories.  We  should  expect  quality  scoring  strategies  to 
reach  completion  faster,  but  we  also  expect  it  to  be  led  astray  more 
easily.  The  sampling  effect  of  the  small  number  of  utterances  in  this 
experiment  does  not  let  us  judge  this. 

Experiment  2:  xNV 

This  experiment  used  the  more  general  middle-out  strategy  (including 
the  collision,  ghost,  and  r.  OSEDIR  heuristics)  described  in  Sec.  E;  again, 
the  experimental  variable  was  the  scoring  method. 

In  this  case,  three  utterances  were  not  completed  and  therefore 
provided  no  useful  comparisons.  The  results  are  summarized  below.  (The 
row  labeled  NTH3  gives  the  sum  of  the  number  of  theories  understood 
correctly  by  methods  Q,  QD,  and  SD.) 


£NV 

SDNV 

SNV 

SDNV 

Correct 

H 

3 

0 

5 

Incorrect 

2 

0 

0 

0 

No  response 

<4 

7 

10 

5 

NTH3 

23 

142 

- 

96 

Again,  shortfall  scoring  produced  no  completions,  with  quality  density 
significantly  mere  breadth-first  than  quality  and  shortfall  density.  In 
this  case,  where  we  can  compare  quality  and  shortfall  density  directly, 
quality  was  much  faster,  but  it  was  not  as  accurate. 
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These  two  experiments  allow  us  to  compare  left-hybrid  and  middle-out 
strategies.  The  results  for  the  more  successful  scoring  methods,  quality 
and  shortfall  density,  are  shown  below.  (As  before,  NTH*!  is  the  summed 
theory  count  for  the  four  utterances  correctly  understood  by  both.) 


LHQNV 

jJNV 

LHSDNV 

SDNV 

Correct 

6 

4 

6 

5 

Incorrect 

1 

2 

2 

0 

No  response 

3 

4 

2 

5 

NTH4 

67 

34 

NTH5  154 

255 

In  both  cases,  the  left-hybrid  strategy  appears  to  get  more  utterances 
correct,  but  in  terms  of  number  of  theories  required,  the  more  general 
middle-out  strategy  appears  to  be  more  efficient  in  the  case  of  quality 
scoring.  If  that  effect  is  real,  and  not  an  artifact  of  our  small  sample, 
then  we  can  offer  no  rationalization  for  it. 

Experiment  3:  .’D+x 

This  experiment  also  used  the  middle-out  strategy  with  shortfall 
density  scoring  and  no  verification;  the  variable  was  the  inclusion  of  the 
three  heuristics  called  collision  events  (C),  ghosts  (G),  and  CHOOSEDIR 
(D),  which  are  described  in  Sec.  E above.  There  are  five,  rather  than 
eight,  combinations,  since  CHOOSEDIR  requires  the  ghosts  heuristic  to  be 
operating . 


The  results  are  summarized  below.  (NTh3  denotes  the  summed  theory 
count  for  the  three  utterances  correctly  understood  by  all  methods.) 


SD+0 

SD+C 

SD+G 

SD+GD 

SD+GDC 

Correct 

3 

3 

3 

4 

5 

Incorrect 

0 

0 

0 

0 

0 

No  response 

7 

7 

7 

6 

5 

NTH  3 

162 

121 

145 

130 

96 

Examination  of  the  individual  utterance  results  in  Appendix  4 shows 

that 

the  inclusion  of  a 

heuristic 

does  not 

always 

guarantee  that 

the  system 

will 

understand  the  utterance  In 

fewer  theories, 

but  the  pooled 

results  above 

(note  especially 

the  series  SD+0 , 

SD+G, 

SD+GD,  SD+GCC ) 

suggest  that 

the 

successively  added  heuristics  produce  improvements  in  both  accuracy  and 
number  of  theories  required. 
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Experiment  U : LHSDxV 

This  experiment  used  the  left-hybrid  strategy  with  shortfall  density 
scoring;  the  variable  was  the  strategy  for  using  the  Verification 
component.  The  strategies,  which  are  described  in  Sec.  E,  are  LHSFNV  (No 
Verify),  LHSDVD  (Verify-at-Do) , and  LHSDVP  ( Verify-at-Pick) . (One 
strategy,  Verify-at-Notice , was  not  run  because  of  lack  of  time;  its 
performance  should  have  been  identical  to  that  of  Verify-at-Pick,  with 
higher  run  times  due  to  many  additional  calls  on  the  Verifier.) 

Each  method  resulted  in  six  utterances  correctly  understood,  two 


incorrect,  and 

two 

no  response. 

The  interesting  results  are  the 

number  of 

theories  and 

the 

CPU  times 

required 

for  the  six  correctly 

understood 

utterances . 

LHSDNV 

LHSDVD 

LHSDVP 

Theories 

183 

141 

100 

Run  time 

( sec 

.)  9443 

9156 

8276 

Even  though  this  small  test  set  shows  no  differences  in  accuracy,  it  does 
show  that  Verification  improves  HWIM's  performance.  The  CPU  time  used  by 
Verification  is  considerable  (about  5 sec.  per  word  verified),  but  the 
decrease  in  the  number  of  theories  more  than  compensates  for  the  extra 
processing  cost.  The  comparison  of  LHSDVD  and  LHSDVP  shows  the  advantage 
of  applying  Verification  as  early  in  the  process  as  possible.  The  extra 
calls  to  Verification  in  LHSDVP  result  in  a better  ordering  of  the  event 
queue,  resulting  in  fewer  incorrect  events  being  processed  and  a net  saving 
of  processing  time. 
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Appendix  1 - Annotated  Phonological  Rules 


Define  phonemes  as  feature  bundles: 

[P 

(PHONEME  (CONS  OBST  LAB  ANT)  )] 

[P 

(PHONEME  (CONS  VCD  OBST  LAB  ANT)  )] 

[F 

(PHONEME  (CONS  OBST  LAB  ANT  CONT)  )] 

tv 

(PHONEME  (CONS  VCD  OBST  LAB  ANT  CONT)  )] 

[W 

(PHONEME  (CONS  HIGH  SONOR  BACK  GLIDE  ROUND  VCD  LAB)  )] 
[M 

(PHONEME  (CONS  NASAL  SONOR  VCD  OBST  LAE  ANT)  )] 

[T 

( PHONFMF  (CONS  COR  OBST  ANT)  )] 

[D 

(PHONEME  (CONS  COR  VCD  OBST  ANT)  )] 

[TH 

(PHONFMF  (CONS  LIST  COR  OBST  ANT  CONT)  )] 

[DH 

( PHONFME  (CONS  DIST  COR  VCD  OBST  ANT  CONT)  )] 

[N 

(PHONFMF  (CONS  COR  NASAL  SONOR  VCD  OBST  ANT)  )] 

[S 

(PHONEME  (CONS  STPID  COR  OBST  ANT  CONT)  )] 

[Z 

( PHONFMF  (CONS  STRID  COR  VCD  OBST  ANT  CONT)  )] 

[SH 

( PHONFMF  (CONS  STRID  COR  HIGH  OBST  ANT  CONT)  )] 

[CH 

(PHONEME  (CONS  STRID  COR  HIGH  OBST  ANT)  )] 

[ ZH 

(PHONEME  (CONS  STRID  COR  HIGH  VCD  OPST  ANT  CONT)  )] 

[JH 

(PHONEME  (CONS  STRID  COR  HIGH  VCD  OBST  ANT)  )] 

[DX 

(PHONEME  (CONS  COR  SONOR  VCD  ANT)  )] 

[NX 

(PHONFMF  (CONS  NASAL  HIGH  SONOR  PACK  VCD  OBST)  )] 

[NY 

(PHONEME  (CONS  NASAL  HIGH  SONOR  VCD  OBST)  )] 

[KY 

(PHONEME  (CONS  HIGH  OPST)  )] 

[GY 

( PHONEME  (CONS  HIGH  VCD  OBST)  )] 

[K 

(PHONEME  (CONS  HIGH  PACK  OBST)  )] 

[G 

(PHONEME  (CONS  HIGH  PACK  VCD  OBST)  )] 

[Y 

(PHONEME  (CONS  HIGH  SONOR  GLIDE  VCD  ANT)  )] 

(PHONFMF  (CONS  LAT  COR  SONOR  VCD  ANT  CONT)  )] 

[ R 

r ( PHONEMF  (CONS  RETRO  COP  SONOR  VCD  ANT  CONT)  )] 

(PHONEME  (CONS  LAT  COR  HIGH  SONOR  BACK  VCD  CONT)  )] 

[ -R 

(PHONEMF  (CONS  RETRO  COR  HIGH  SONOR  BACK  VCD  CONT)  )] 
[HH 

(PHONEME  (LOW  ASP  GLIDF ) )] 

10 

(PHONEMF  (LOW  GLIDE)  )] 

[HW 

(PHONEME  (HIGH  BACK  GLIDE  ROUND  LAB)  )] 

[ URP 

(PHONEMF  (CONS  UNREL  OBST  LAB  ANT)  )] 

['JRT 

(PHONEME  (CONS  UNREL  COR  OBST  ANT)  )] 
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[ UPK 

[URB 

[URD 

[URG 

[IY 

[IH 

[FY 

[FH 

[EA 

[AE 

[IX 

[AX 

[UW 

[UH 

[OW 

[AO 

[AW 

[AY 

[OY 

[AA 

[AH 

[FR 

[EL 

[EN 

[EM 

[ AXR 

[ HIY 

[HIH 

[UIX 

[ HUW 

[HUH 

[UAX 

[TR 

[DR 

[SR 

[ZR 

[UY 


(PHONEME  (CONS  UNREL  HIGH  PACK  OBST)  )] 

( PHONEMF  (CONS  UNREL  VCD  OBST  LAB  ANT)  )] 

(PHONEME  (CONS  UNREL  COR  VCD  OBST  ANT)  )] 

(PHONEMF  (CONS  UNREL  HIGH  BACK  VCD  OBST)  )] 

(PHONEME  (VOWEL  ATR  HIGH  SONOR  VCD  ANT)  )] 

( PHONFMF  (VOWEL  HIGH  SONOR  VCD  ANT)  )] 

(PHONEME  (VOWEL  ATR  SONOR  VCD  ANT)  )] 

(PHONEME  (VOWEL  SONOR  VCD  ANT)  )] 

(PHONEME  (VOWEL  ATR  LOW  SONOR  VCD  ANT)  )] 

(PHONEME  (VOWEL  LOW  SONOR  VCD  ANT)  )] 

(PHONEME  (REDUCED  VOWEL  HIGH  SONOR  VCD  ANT)  )] 

(PHONEME  (REDUCED  VOWEL  SONOR  VCD)  )] 

(PHONEME  (VOWEL  ATR  HIGH  SONOR  BACK  ROUND  VCD)  )] 
(PHONEME  (VOWEL  HIGH  SONOR  BACK  ROUND  VCD)  )] 

(PHONEMF.  (VOWEL  SONOR  BACK  ROUND  VCD)  )] 

(PHONEME  (VOWEL  LOW  SONOR  BACK  ROUND  VCD)  )] 

(PHONEME  (DIPHTHONG  VOWEL  LOW  SONOR  BACK  ROUND  VCD)  )] 

(PHONEME  (DIPHTHONG  VOWEL  LOW  SONOR  BACK  VCD)  )] 

(PHONEME  (DIPHTHONG  VOWEL  SONOR  BACK  ROUND  VCD)  )] 
(PHONEME  (VOWEL  LOW  SONOR  BACK  VCD)  )] 

(PHONEME  (VOWEL  SONOR  BACK  VCD)  )] 

(PHONEME  (VOWEL  RETRO  COR  HIGH  SONOR  BACK  VCD)  )] 
(PHONEME  (VOWEL  LAT  COR  HIGH  SONOR  BACK  VCD)  )] 

(PHONEME  (VOWEL  COR  NASAL  SONOR  VCD  ANT)  )] 

(PHONEME  (VOWEL  NASAL  SONOR  VCD  LAB  ANT)  )] 

(PHONEME  (VOWEL  RETRO  COR  SONOR  VCD)  )] 

(PHONEME  (VOWEL  ATR  HIGH  SONOR  ANT)  )] 

(PHONEME  (VOWEL  HIGH  SONOR  ANT)  )] 

(PHONEME  (REDUCED  VOWEL  HIGH  SONOR  ANT)  )] 

(PHONEME  (VOWEL  ATR  HIGH  SONOR  BACK  ROUND)  )] 

(PHONEME  (VOWEL  HIGH  SONOR  BACK)  )] 

(PHONEME  (REDUCED  VOWEL  SONOR)  )] 

(PHONEME  (CONS  VCLESS  ASP  RETRO  COR  OBST  ANT)  )] 
(PHONEME  (CONS  RETRO  COR  VCD  OBST  ANT)  )] 

(PHONEME  (CONS  VCLESS  RETRO  STRID  COR  OBST  ANT  CONT)  )] 
(PHONEME  f CONS  RETRO  STRID  COR  VCD  OBST  ANT  CONT)  )] 
(PHONEMF  (VOWEL  ATR  HIGH  SONOR  ROUND  VCD  ANT)  )] 


i 


I 
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Rule  tree  used  for  phonological  rule  expansion  of  the  dictionary: 

ZUFL1ST  ((OPT  Z3  .7  .F) 

(OPT  Z3A  .7  .5) 

(OPT  Z‘4  .7  .f) 

OPT  Z 44 A .7  .5) 

OPT  Z5  .7  .5) 

OPT  ZFA  .7  .5) 

OPT  Z6  .7  .5) 

OPT  Z6A  .7  .F) 

(OPT  Z7  .7  .5) 

OPT  zfc  .7  .5) 

(OPT  Z8A  .7  . 5 1 
OPT  Z1  .7  .F) 

OPT  Z2  .7  .5) 

(OPT  FR1) 

R446  P16  R16.1  (OPT  Z 16  1.0  1.0) 

(OPT  ZF6  1.0  .2} 

(OPT  R 17  .8  1.0) 

(OPT  Z 1 8 .8  1.0) 

(OPT  Z 10  .8  1.0) 

(OPT  ZFfi) 

(OPT  RIO  .8  1.0) 

R10B 

(OPT  Z22.I4) 

(OPT  122.  1) 

OPT  122.2) 

(OPT  Z22.3  -1  1.0) 

( IF  (OPT  R26  1.0  .6) 

THEN  R27 ) 

(OPT  R’2  1.0  1.0) 

(ELSE  (OPT  R’2E)) 

OPT  Z60) 

(IF  (OPT  RFO  1.0  .5) 

THEN  RFF ) 

(OPT  Z HA  1.0  .2) 

(ELSE  (OPT  Z 1 1B  1.0  .2) 

(''LSE  (OPT  Z IRC  1.0  .2) 

( ELSF  (OPT  ZI’D  (.0  .2)))) 

(OPT  Z13t  1.0  .5) 

(OPT  ZF7 ) 

(OPT  R30.Z  . F 1.0)) 


Rule  tree  used  for  APR  rule  expansion  of  the  dictionary: 

APRLIST  ((OPT  APRND  .5  1.0) 

(OPT  APPND2  . F 1.0) 

APR  1 APR  1 A APR2  APR2A  APR’  (IF  APRU  THEN  R55) 
APR  IF  APR  16  APR  10  APR  1 1 APR12  APR  1 7 (OPT  APR  1 44 ) 
APR  17  APR  1 7 A A P R 1 7 P ) ) 
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* Rule  P-EST.A  through  R-ING  are  rules  used  for  deriving 
**  inflections  from  the  uninflected  base  forms,  depending 


Listing  of  the  rules: 

•*  Rule  P-EST.A  throu 

*•  inflections  from  t 

••  features  of  the  last  segment. 

! 


[R-FST.A 

(RULE  ( 


[R-FST.P 

(RULE  (( 

[ R-FST.C 

(RULE  (( 


[R-EST 


[R-ER. 

[R-ER 


S (EITHER  P OR  T OR  K)  -EST) 

i?” 

( : IX  !-ST!))  )] 

NX  -F.ST ) 

•I  (:  • IX  !-  S T)))  )] 

U CONS) 


[ R-ED2 
[R-ED3 
[ R-S1 

[ R-S2 
[F-S3 


[R-ING 


(• 

(0  •) 

(:  IX  !-  ST)))  )] 

(RULE 

( -EST  (:  • IX  t-  S T))  )] 

A 

(RULE 

((• 

S (EITHER  P OR  T OR  K)  - 

<• 

1?  •) 

( : AXR  !-)))  )] 

B 

(RULE 

((• 

NX  -ER) 

c 

(• 

•I  (:  r AXR  !-)))  )] 

( RULE 

((• 

(+  CONS)  -ER) 

(* 

(0  •) 

(:  AXR  !-)))  )] 

(RULE 

(-EF 

1 (:  • AXR  !-))  )] 

.A 

( RULF, 

( • 

S T -ED) 

• 

l?  #) 

. B 

(:  IX  !-  D)))  )] 

(RULE 

((• 

(EITHER  T OR  D)  -ED) 

(« 

(0  •) 

(:  IX  !-  D)))  )] 

(RULE 

( -EC 

) T / (+  CONS  - VCD)  — ) 

(RULE 

( -EL 

> D)  )] 

(RULE 

((• 

(EITHER  S OR  Z OR  SH  OR  ' 

-S) 

(• 

(0  •) 

(:  IX  ! - Z) ) ) )] 

(RULE 

(-S 

S / (+  CONS  - VCD)  — ) ) 

(RULE 

(-S 

Z)  )] 

.A 

(RULE 

((• 

S (EITHER  P OR  T OR  K.)  - 

(• 

i; #) 

; , b 

( : IX  !-  NX)) ) )] 

(RULE 

((• 

NX  -INC) 

[ Q 

(* 

•I  (:  • IX  ! - NX)))  )] 

(RULE 

((• 

(+  CONS)  -ING) 

(• 

0 •) 

( : IX  !-  NX)))  )] 

(RULE 

(-ING  (:  • iy  ! - NX))  )] 

ING) 
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#*  1 \ tt)rou£h  Z2  are  used  for  vowel  reduction.  Note 

•*  thpeiic(n  vowels  are  not  allowed  tc  be  reduced  if  they  appear 
as  the  last  segment  of  a word.  1 

[ Z 3 

(RULF  (IY  IH  / — 1 0 ( EITHER  (*£#13  (♦  CONS)) 


OR 


[ Z3A 
[ZH 


l 


(•  * (♦  CONS))) 


VOWEL))  )] 


(RULE  (IY  IH  / # (#  0 3 (+  CONS)) 

-y  !0  (#  0 2 (+  CONS)) 

(RULE  ( EY  EH  / — !0  (EITHER  (*(#13  U CONS)) 

* ) 


OR 


[ZHA 

[Z5 


l 


(*  • (+  CONS))) 


VOWEL))  )] 


(RULE  (EY  EH  / # (#  0 3 (+  CONS)) 

--  ! 0 (#02  (+  CONS)) 

#)  ) J 

(RULE  (UW  UH  / — !0  (EITHER  (»  (#  1 3 (+  CONS)) 

OR 

(*  * (-  CONS))) 


[Z5A 

[Z6 


[Z6A 

[Z? 
r Z8 


♦ VOWEL))  )] 

(RULE  (UW  UH  / If  (it  0 3 U CONS)) 

--  ! 0 (#  0 2 (+  CONS)) 

«)  )] 

(RULE  (OW  AH  / — !0  (EITHER  (»  |(M  1 ( + CONS)) 

OR 

(*  * (+  CONS))) 

(+  VOWED)  )] 

(RULE  (OW  AH  / It  (fl  0 3 (*  CONS)) 

“ JO  (#  0 2 ( + CONS)) 

(RULE  ((*  ER  ! 0 ) 

(•  AXR  !-))  )] 

(RULE  (AW  AA  / — !0  (EITHER  (*£#13  (+  CONS)) 


[Z8A 

[Z1 
[ Z2 


l 


n 

OR 

(*  * (+  CONS))) 


VOWEL))  )] 


(RULE  (AW  AA  / # (#  0 3 (+  CONS)) 
-y.j ! 0 (#  0 2 (♦  CONS)) 


(RULE  ((•  IH  ! 0 ) 

(*  IX  !-))] 


(RULF  ((*  (EITHER  EH  OR  AE  L.,  AA  OR  AO  OR  AH  OR  UH) 
(•  AX  !-))  )] 
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H Front  rounding  rule,  change  UW  to  a front  rounded  vowel  UY. 
••  e.g.  "New  York"  N UW  Y AO  R K =>  N UY  Y AO  R K 


[FR1 

(RULE  CJW  UY  / (EITHER  (+  COR  - LAT  - RETRO) 

OR  Y) 

(EITHER  ! 2 OR  ! 1 OR  !0) 

(OFT  •) 

(EITHER  (+  COR  - LAT  - T YTRO) 

OR  Y))] 


**  This  rule  is  used  to  clean  up  after  the  vowel  reduction  rules. 
**  It  deletes  one  of  the  schwas  if  ttwo  appear  in  sequence. 


[RU6 

(RULE  ((•  AX  !-) 

0 / AX  ! ) )] 


•»  Syllabification  of  R. 

•*  e.g.  "for"  F AX  R = > F AXR 


[ R 1 6 


(RULE  ( 


AX  !-  R) 

AXR  ! - 0))] 


**  Syllabification  of  R across  syllable  boundary. 
••  e.g.  "arrange"  AX  R EY  N JH  =>  AXR  EY  N JH 


[R16.1 


(RULE  ( 


[.* 


AX  ! - * R) 

AXR  ! - • 0))] 


**  D'-syllabify  AXR  when  It  appears  between  vowels 

••  e.g.  "covering"  K AH  V AXR  IX  NX  =>  K AH  V AX  R IX  NX 

[ Z 1 6 

(RULE  ((*  AXR  ! 

(•  AX  !- 
/ — ( + 


- •) 

(:  » R)) 
VOWEL))  )] 


**  Transitional  glide  insertion  fc  W 
••  e.g.  "going"  G OW  IX  NX  =>  G OW  W IX  NX 

[Z56 

(RULE  (0  W / (EITHER  OW  OR  AW  OR  UW) 
(EITHER  !2  OR  !1  OR  !0) 

* — (+  VOWEL  + ANT))] 


**  Syllabification  of  L. 

••  e.g.  "travel"  T R AE  V AX  L =>  T R AE  V EL 
[H17 

(RULE  ((•  (EITHER  AX  OR  IX) 

! - L) 

(^  EL  ! - 0) 

/ --  (EITHER  (+  CONS) 

OR  *))  )] 
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**  Syllabification  of  N. 

“ e.g.  "London"  L AH  N DAXN  =>  L AH  N DEN 

[ZIP 

( RULE  ((«  ( EITHER  AX  OR  IX) 

! - N) 

(«  EN  ! - 0) 

/ 

(+  CONS) 

[EITHER  (+  CONS) 

OR  it))  n 


**  Syllabification  of  M. 

»«  e.g.  "item"  AY  T AX  M =>  AY  T EM 

[ Z 1 9 

(RULE  ((«  (EITHER  AX  OR  IX) 

! - M) 

(«  EM  ! - 0) 

/ 

(+  CONS) 

(EITHER  (+  C^NS) 

OR  #))  )] 


**  Glottal ization  of  T. 

««  e.g.  "gotten"  G AA  T EN  =>  G AA  0 EN 

[ Z58 

(RULE  (T  0 / !2  « — (EITHER  EN  OR  EL))] 


**  The  next  two  rules  are  for  homorganic  stop  insertion. 
*«  e.g.  "since"  S IH  N S =>  S IH  NTS 

[RIO 

(RULE  (0  (+  OBST  + CONS  + ANT  - CONT  (X) 

LAB 

(Y) 

COR 

(Z) 

VCD) 

/ 

U NASAL  X LAP  Y COR) 

(OPT  !-) 

[opt  ») 

(+  CONT  - SONOR  Z VCD  (X) 

LAB 


(PULE  ((»  T * SH) 

(*  0 » CH))  )] 


**  Palatalization  rule  that  inserts  Y before  UW. 
»*  e.g.  "assume"  AX  S UW  M =>  AX  S Y UW  M 

[Z  22. H 

(RULE  (0  Y / (EITHER  S OR  N OR  JH) 

— UW)  )] 
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L : i :%-i  4 3We 


....  . 


**  Palatalization  rule  for  fricatives 

••  e.g.  "misuse"  M IH  S Y UW  Z =>  H IH  SH  Y UW  Z 

[ Z22 . 1 

(RULE  ((EITHER  S OR  Z) 

( + HIGH) 

/ 

(+  VOWEL  + HIGH) 


;opt  ») 

+ HIGH  + 
OPT  •) 

;+  VOWEL)) 


ANT  - CONS) 

)] 


**  Palatalization  rule  for  Y-IY-SCHWA  sequence. 
**  e.g.  "California"  K AE  L F AO  R N IY  AX  => 


K AE  L F AO  R N Y AX 


[Z22.2 


( 


RULE  ((»  IY  (EITHER  ! 1 OR  ! 0 ) 

•) 

(»  Y 0 0) 

/ 

(+  CONS  + COR) 

(OPT  ») 

("EITHER  IX  OR  AX))  )] 


**  Palatalization  rule  for  stops 

**  e.g.  "Tuesday"  T Y UW  Z D EY  =>  CH  Y UW  Z D EY 
[Z22.3 

(RULE  ((EITHER  T OR  D) 

(♦  HIGH  ♦ STRID) 

/ — (OPT  •) 

Y 

'OPT  **) 

+ VOWEL) 

(EITHER  ! 2 OR  ! 1 OR  ! 0) ) ] 


##  The  next  two  rules  are  for  HW  unrounding. 
••  e.g.  "which"  HH  W IH  CH  =>  W IH  CH 

[ R 26 

(RULE  ((»  HH  W) 

(•  0 HW))  )] 

[R27 

(RULE  (HW  W)  )] 


»•  RUH  reduction  rule  (Case  A):  RUH  is  preceded  by  at  least  one 
##  consonant  and  followed  by  a vowel . 

»•  e.g.  "propose"  P R AX  P OW  Z =>  P AXR  P OW  Z 


[ R 32 


:RULE  ((»  R (EITHER  AX  OR 

!-) 

(•  AXR  0 !-) 

/ 


U CONS) 

— $ (+  VOWEL) 
(EITHER  ! 2 OR  ! 1))] 


IX) 
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**  RUH  reduction  rule  (Case  B):  RUH  is  following  a stressed  vowel 
**  and  zero  or  more  consonants  (but  not  an  NX) 

e.p.  "hundred"  HH  AH  N D R IX  D =>  HH  AH  N D AXR  D 


• « 


[ R3?B 


(RULF  ((• 


R ( EITHRR 

!-) 

AXR  0 !-) 


AX  OR  IX) 


(* 

/ 

(+  VOWEL) 

(EITHER  !2  OR  !1) 
(#  0 2 (+  CONS)) 
OPT  •) 

(#  0 2 (+  CONS)) 


(FITHER  (-  NASAL)  OR  (-  BACK)  OR 
(-  SONOR)  OR  (-  HIGH)  OR 
(-  VCD)  OR  (-  OBST) ) ] ) 


**  Stop  deletion  rule 

**  e.g.  "arrange"  AX  R EY  N JH  =>  AX  R EY  N ZH 
[Z60 

(RULE  (JH  ZH  / N — #)] 


• * rule;  applies  t0  a schwa  when  it  is  prece< 

stressed  syllable  and  followed  by  a unstressed  syllable, 
resulting  extra  syllable  boundary  is  removed  by  R55. 

*•  e.g.  "original"  AO  R 1H  G IX  N EL  =>  AO  R IH  G N EL 


The' 


[R50 

(RULE  ((•  (EITHER  AX  OR  IX) 

!-) 

(*  0 0) 

(+  VOWEL) 

! 2 

jf  0 2 (+  CONS)) 

(H?(t  CONS  - GLIDE)) 

--  * (EITHER  N OR  M OR  R OR  L) 
+ VOWEL) 

(EITHER  ! 0 OR  !-))) 


*# 


Rules  Z1 3A-E  are 
e.g.  "enter"  EH 


arious  forms  of 
N T AXR  =>  EH  N 


the  flapping  rule 
DX  AXR 


for  dental  stops. 


[Z13A 


(RULE  ( ( EITHFR  T OR  D) 
DX  / (+  VOWEL) 

! 2 

OPT  R) 

(OPT  ») 


(OPT  ») 

(+  VOWEL) 
(EITHER  ! 0 
(RULE  ((EITHER 
DX  / ( + 

! 1 

(OPT  R) 
(OPT  «) 


OR  !-))] 
T OR  D) 
VOWEL) 


(OPT  *) 

(+  VOWEL) 

(EITHER  ! 9 OR  !-))  )] 
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[ z nc 

(RULE  ((FITHER  T OR  D) 

DX  / ( + VOWEL) 

! 0 

(OPT  R) 

(OPT  •) 

(OPT  •) 

(+  VOWEL) 

(EITHER  ! 0 OR  !-))  )] 

[Z13D 

(RULE  ((EITHER  T OR  D) 

DX  / ( + VOWEL) 

! - 

(OPT  R) 

(OPT  ») 

Fopt  *) 

(+  VOWEL) 

!-)  )] 

[ZHE 

(RULE  (T  DX  / (+  VOWEL) 

(EITHER  !2  OR  ! 1 f R ! 0 OR  !-) 
N 

(OPT  •) 

(OPT  *) 

+ VOWEL) 

(EITHER  ! 0 OF  !-))] 


•*  Flapping  rule  for  N. 

••  e.g.  "any"  EH  N IY  =>  EH  TX  IY 

[Z57 

(RULE  (N  TX  / ! 2 * — IY  !0  (OPT  *) 
(EITHER  U CONS)  OR  //))] 


**  schwa  devoicing  rule:  applies  to  schwa  between  two  voiceless  stops. 
*•  e.g.  "multiply"  M AH  L T UIX  P L AY  =>  y AH  L T P L AY 

[RRO.Z 

(RULE  ((EITHER  AX  OR  IX) 

(-  VCD) 

/ 

(EITHER  (-  VCD  + CONS  - CONT  + OBST)  OR  JH) 

( OPT  «) 

— ! - (OPT  «) 

(-  VCD  + CONS  - CONT  + 083T) 

K VOWEL))] 

(R55 

(RULE  (•  0 / — (#  0 2 U CONS))  *)] 


*•  nasal  deletion  rule:  if  APR  could  detect  nasalized  vowels,  this  would 
*•  be  replaced  by  two  legitimate  phonological  rules,  the  first  of 

which  would  nasalize  the  vowel  optionally  and  the  second  of  which  wou 
••id  delete 

••  the  nasal  optionally  after  a nasalized  vowel. 

*•  e.g.  "spent’’  S P EH  N T =>  S P EH  T 

[ APRND 

(RULE  (N  0 / !2  — (OPT  «) 

U CONS))] 


••  Nasal  deletion  rule  for  i. 

••  3 . g . "assumption"  AX  S AH  M P SH  EN  =>  AX  S AH  P SH  EN 
[ APRND2 

(rule  (H  f/cife-;  m,;> 
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**  rulos  APH1  through  APRR  arc  allophon<c  ruler  for  vowels. 
««  e.p.  "trip"  T R 1H  P =>  T P PTH  P 


[ APR  1 
[ APR  1 A 
[APR? 

[ APR2A 
[ A PR  3 


(RULf- 

(IH 

RIH 

/ 

R — !?)] 

( RULE 

(1H 

RIH 

/ 

— ! 2 R)  )] 

(RULE 

( IH 

LIH 

/ 

L — ! 2)  ] 

(RULE 

(IH 

LIH 

/ 

— 12  L)] 

( RULE 

(IX 

INX 

/ 

— !-  NX)] 

**  voiceless  schwa  deletion  (The  APR  currently  does  not  detect  voiceless 
**  schwa) 

**  e.g.  multiply 

t APR1! 

(RULE  ((•  (EITHER  UIX  OR  UAX) 

!-) 

(«  0 0))] 


•*  rules  APR10  through  A PR  1 3 introduces  allophones  for  T. 
[ APR  10 

(RULE  (T  TG  / — (EITHER  R OR  W OR  L OR  Y))] 

[ APR1 1 

(RULE  (T  ST  / (+  STRID) 

(+  VOWEL))] 

[ APR  1 2 

(RULE  (T  TV  / — (+  VOWEL))] 

[APR1R 

(RULE  (T  TS  / — (EITHER  S OR  Z OR  SH  OR  ZH ) ) ] 


**  allophone  for  CH 


[APR  111 

(RULE  (CH  ( : TS  TCH))] 


««  APR  rule 


for  converting  AX  to  IX  based  on  context. 


[ APR15 

(RULF 


**H ) 


**H ) ) ] 


(AX  IX  / 

(EITHER  T OR  D OR  N OR  K OR  G OR  NX  OR  S OR  Z OR  CH  OR  J 
— ! - (OPT  •) 

(EITHER  T OR  D OR  N OR  K OR  G OR  NX  OR  S OR  Z OR  CH  OR  J 


**  allophone  for  back  K 


[ APR  1 6 

(RULE  (K  KA  / — (EITHER  (+  BACK  + VOWEL) 

OR  AX  OR  AXR  OR  R OR  W OR  L))] 


««  rules  APR  17 


through  APR  1 7 B create  allophone  for  nasals. 


[ APR  17 

(RULE  (N  YN  / 

(EITHER 
(EITHER 
(OPT  «) 

[APR  1 7 A 

(RULE  (M  YM  / 

(EITHER 
EITHER 
(OPT  «) 

[apri7Prule  (nx  n , A0 


IY  OR  EY  OR  R OR  ER  OR  AXR) 
12  OR  ! 1 OR  !0  OR  !-) 

— )] 


IY  OR  EY  OR  R OR  ER  OR  AXR) 

!?  OR  ! 1 OR  ! 0 OF  !-) 

~)  )] 

(EITHER  !2  OR  !1  OR  !0  OR  !-)  — )] 
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Program  to  Generate  Across  Word  Phonological  Rules 


static  { 
S2V  wl 

SVwl  : 

HBVwl 
WkVwl 
Vwl  :: 

Stp  : = 
URStp 

Cnst  1 

PreUY 

CrCon 

Cnst2 

Cnst3 

CnstH 


Outjfn  :=  nil 


//  Stress  2 vowels 
:=  table  IQ. 

” IY ! 2"  , "IH!2"  , " EH ! 2” , "AE!2",  "AA!2",  "AH ! 2" , "AO!2", 
"UH!2",  "UW! 2" , " ER ! 2" , "EY!2"  "0W!2",  "AW!2",  "AY!2", 
"OY!2",  "UY  ! 2"  , "EA!2",  "LIH!2\  "RIHf  2" 


//  Stressed  vowels 
= table  38, 

" IY 1 2" , " IH ! 2",  "EH!2" , "AE!2"  " AA f 2" , "AH!2"  "AO!2". 

"UH  ! 2"  , "UW!  2"  , "ER!2"  , "EY!2%  "OW!2%  "AW!2*,  " A Y ! 2 " , 
"OY ! 2" , "UY ! 2" , "EA!2",  "LIH!2* ,"niH!2* ,"IY!  1"  , "IH! 1" , 
" EH ! 1 " , "AE!  1" , "AAM",  "AH!1",  "A0!1",  "UHM",  "UWM", 
" ER ! 1 " , " EY ! 1 " , "OW! 1 " , "AW!  1"  , "AYM",  " OY  ! 1 " , "UY!1", 
"EA! 1",  "LIH ! 1 ",  "RIH!1" 

//  "tgh  back  vowels  for  W-deletion 
:=  table  3, 

"AW! 2" , "OW! 2" , "UW!2" 

//  Weak  vowels  for  Flapped  T rule 
• — tablG 

"IH ! 1 " , "Ih!o",  "AX!-",  "IX!-" 


//  List  of  all  vowels 


table 
"IY !0"  , 

66, 

" IY  1 1 " , 

" IY I 2" , 

"IH !0" , 

"IH!  1", 

" IH ! 2" , 

" EH ! 0 " 

" EH  ! 1 " , 

"EH! 2", 

"AE!0" , 

"AE! 1", 

"AE ! 2" , 

"AA ! 0" , 

"AAM" 

" AA ! 2"  , 

"AH ! 0" , 

"AHM"  , 

"AH! 2", 

"AO ! 0"  , 

"AOM", 

"AO! 2" 

"UH ! 0" , 

"UH! 1", 

"UH ! 2" , 

"UW! 0" , 

"UWM", 

"UW! 2"  , 

"AX!-" 

"ER ! 0" , 

"ER! 1", 

"ER!2" , 

"EL!-", 

" EY ! 0 " , 

"EY! 1", 

"EY ! 2" 

"OWIO" , 

"OW! 1", 

"OW! 2" , 

"AW ! 0" , 

"AW! 1 " , 

"AW! 2" , 

" A Y ! 0 " 

"AY!  1", 

"AY! 2", 

"OY ! 0" , 

"OY! 1", 

"0 

"RIH ! 1 " , 

"RIH! 2" 

"INX ! 


//  These  stops  can  be  changed  to  URPlosives 
: table  6, 

up,,  ii Tn  .* k " , "B",  "D",  "G" 

:=  fable  6, 

"URP" , "URT",  "URK" , "URB",  "URD",  "URG" 

//  Right  context  for  UR  stop  rule 
:=  table  1^, 

up,.  » XG " , "TV"  , "TS",  "K",  "B",  "D", 

"G",  "F",  "TH",  "JH",  "CH",  "DH", 


//  Pre  UY  context 
:=  table  6, 

"TV",  "ST",  "D",  "HH",  "N",  "Y" 

//  post  UY  context 
:=  table  8, 

"D",  "TV",  "TG",  "TS",  "S",  "N" , "TH" , 


ii  Y« 


//  V deletion 
:=  table  7, 

"M",  "B",  ’,D",  "DH",  "F",  "P",  "HH" 

//  DH  deletion 
:=  table  11, 

"N",  "Z",  "CH",  "TCH",  "T",  "JH",  "ZH",  "TH" , 

titytf  ^ wgfi  tt^.1! 

//  D Deletion 
:=  table  10, 

"DH",  "TG",  "TV",  "TS",  "CH",  "S",  "L" , "W" 


JH&! 
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//  T Deletion 
UVKrie  : = table  H . 

"S",  "F",  *DH",  "TH" 


//  Geminate  reduction 
GCnst  :=  table  12, 

"P",  "K",  "B",  "D",  "G",  "F" 
"S",  "Z",  "SH" , "TH" , "M" , "N" 


//  HH  Deletion 
Strdl  :=  table  8, 

"S",  "SH",  "CH",  "TCH", 


"DH", 


"P", 


"T" , 


"K" 


//  context  for  creating  allophones  of  T 
//  ST  allophone 
Strc12  :=  table  1, 

"S" 


NStrd 


Strd3 
Gld  : 


//  TV  allophone 


= table  97. 

"IYI0", 

" I Y f 1 " , 

"IY!2" , 

"IH ! 0" , 

"IH! 1", 

"IH! 2" , 

” EH ! 0 " , 

"EH! 1", 

"EH! 2", 

"AE!0" , 

"AE! 1", 

"AE! 2" , 

"AAIO", 

"AAM", 

" AA ! 2" , 

" AH ! 0 " , 

"AH ! 1"  , 

"AH! 2", 

" AO ! 0 " , 

"AO! 1" , 

"A0! 2", 

"UH!0" , 

"UH! 1", 

"UH ! 2" , 

"UWIO", 

"UW! 1", 

"UW! 2" , 

"AX!-" , 

"ER ! 0" , 

"ER! 1", 

"ER 1 2" , 

"EL!-", 

"EY ! 0" , 

" EY ! 1 " , 

"EY ! 2" , 

"OW! 0" , 

"OW! 1", 

"OW! 2" , 

" AW!  0" , 

",.W!  1", 

"AW! 2", 

" AY ! 0" , 

"AY! 1", 

"AY  1 2", 

"OY ! 0" , 

"OY ! 1" , 

"OY ! 2" , 

tit  n 

"NX* , 

" W " , 

"R", 

"Y" 

"N", 

"M", 

n pit 
r * 

"K", 

"B" , 
"DHA, 
"URP*5 , 

"D", 
"HH"  , 
"URTa, 

"G", 
"DX" , 
"URK", 

, "UY ! 0" , 

"F" 

" TH  " , 

"V", 

"ENI-", 

"EH!-", 

"ENX" , 

"URB", 

" UY ! 1 , 

"URD", 
"UY !2" , 

"URG", 
"EA ! 0" , 

"IX!-'', 

"AXR ! 

"EA! 1", 

"EA ! 2" , 

"UAX!-" 

, "TX" , 

"Q", 

" TS  " . 

"liAm" 

"UIX ! -" 
,"LIH! 2" 

ncTtt 

’"rihIo" 

ii'pyit 

,"rih(i" 

"TG" , 

, "RIH! 2" 

"INX !-" 

//  TS  allophone 
:z  table  3, 

"S",  "SH",  "Z" 

//  TG  allophone 
table  5, 

"R",  ,.w..  nL«  «■  Y” 


//  left  context  for  YN  and  YM  allophones 
NasB  table  11, 

“ll  p II 

"Axft!-" 

"ER !0" , "ER ! 1" , "ER ! 2" , 

"IY 10" ,"IY II " , "IY ! 2" , 

"EY 10" , "EY! 1 " , "EY ! 2" 


let  WtStrs(String,nil  repname  20)  be  { let  Strings  :=  lv  String 
Writes (Out Jfn .String ) 
for  i :=  1 to  NumbArgs()-1  do 
{ B0UT(0utJfn,$*s)  ; WriteS(OutJfn .Strings ! i) 

WriteS(OutJfn ,"*c*l" ) 
return 

) 


and  Start()  be  { WriteS("Rules  Generated  are  to  go  to") 
OutJfn  :=  CreateOutput( 1 ) 

//  unreleased  stops  before  obstruents 
for  i :=  1 to  StplO  do 
{ for  1 ::  1 to  Unstl  ',0  do 

t WtStrs(Stp!i,"#",Cnst1 ! j ,"  = >" ,URStp ! i ,Cnst 1 ! j) 
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//  V - deletion  (mostly  before  labials) 
for  i :=  1 to  Cnst2'0  do 
{ WtStrsC'V  #"  , Cnst2 ! i , " = , Cnst2l  i ) 

} //  This  is  an  APR  rule  for  when  it  is  called  obstruent 
Wt£trs( "V  # M =>  V") 

//  DH  - deletion  (after  dentals,  palatals,  M,  - 
for  i :=  1 to  Cnst3l0  do 
{ WtStrs(Cnst3!i,"#  DH  =>",Cnst3li) 

//  HH  - deletion  (after  fricatlon  or  aspiration)  //  Is  really  a 
which  applies  to  HAVE, HAS, HAD  etc. 
for  i :=  1 to  Strdl 10  do 

{ ^ WtStrs ( Strdl ! i , "#  HH  AE ! 2 =>" , Strdl ! i , "AE! 2" ) 

//  D - deletion  (following  N,  preceding  many  dental  consonants) 
for  i :=  1 to  Cns 1 i 0 do 
{ ^ WtStrs ("N  D #",Cnst^!i,"=>  N",CnstU|l) 


//  W - deletion  (following  HBVwl) 
for  i :=  1 to  HBVwl |0  do 
{ WtStrs ( HBVwl  W = >", HBVwl i ) 


111 - deletion  (between  S and  UVFric 
for  i :=  1 to  UVFric |0  do 
{ ^ WtStrs ("S  T #" , UVFric ! i , " = > S",UVFric|i) 


//  T - deletion  rule  (special  case  for  S) 

WtStrsC'S  T # S =>  S") 

II  Flapped  T rules  - right  now,  following  a Vwl,  preceding  a Vwl. 
for  i :s  1 to  Vwl|0  do 
{ for  j : = 1 to  Vwl ! 0 do 

II  Word  ending  T or  D - allow  for  any  vowel. 

{ WtStrs( Vwl j i , "T  #" , Vwl j 1 , "=>" , Vwl j i ,"DX" , Vwl ! 1 ) 

WtStrs( Vwl ! i , "D  , Vwl ! j , " = >" , Vwl ! i , "DX" , Vwl ! j ) 


for  j :=  1 to  WkVwllO  do 

//  Word  initial  T - allow  following  S2Vwl  and  preceding  WkVwl. 
{ for  i :=  1 to  S2Vwl|0  dc 

WtStrs(S2Vwl ! i "#  TV" , WkVwl ! J ,"=>" ,S2Vwl ! i ,"DX" , WkVwl | j ) 

//  for  word  initial  T or  D preceded  by  R,  require 
//  preceded  by  SVwl  and  followed  by  WkVwl 
for  i :=  1 to  SVwl'O  do 


{ WtStrs ( SVwl !i,"H  # TV", WkVwl ,'j 
^ WtStrs( SVwl ! i R # D" , WkVwl ! j , 


A 


"=>" .SVwl ! i ,"R  DX" 
=>" ,SVwl ! i ,"R  DX", 


.WkVwl! 1 
WkVwl lj) 


) 


) 


II  Palatization  rules. 
WtStrs("S  # Y =>  SH") 
WtStrsi "S  « SH  =>  SH") 
WtStrs  "Z  « SH  =>  SH") 
WtStrs("D  # Y =>  JH") 


//  Geminate  reduction 

for  i :=  1 to  GCnstlG  do 
t WtStrs ( GCnst |i ,"#" ,GCnst ! i , ,GCnst ! i) 

} //  specific  geminate  reduction  rules  for  T a.llophones 
WtStrs("T  # TV  =>  TV") 

WtStrs  "T  6 TG  =>  TG") 

WtStrs( "T  # TS  =>  TS") 
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rule 


WtStrs 

"T 

tt 

D => 

D") 

WtSt  rs 

"D 

II 

TV  => 

TV 

WtStrs 

" I) 

II 

TG  => 

TG 

WtStrs 

"D 

H 

TS  => 

TS 

WtStrs 

"P 

H 

B => 

B") 

WtStrs 

"B 

II 

P => 

P") 

WtStrs 

"K 

# 

G => 

G" 

WtStrs 

"G 

II 

K => 

K") 

//  these  are  APR  geminate  reduction  rules. 
WtStrs("Z  II  S =>  S") 

WtStrs! "SH  II  S =>  S") 

WtStrs ("CH  II  SH  =>  CH") 

WtStrs("N  II  M =>  N") 

WtStrs("M  II  N =>  M" ) 

//  The  following  rules  allow  the  5 allophones  of  T 
//  preceded  by  StFRIC 


for  i : = 1 to  Strd2|0  do 
r j :=  1 to  VwliO  do 

Wt3trs(Strd2i  i ,"T  It"  ,Vwl ! j ,'•  = >"  ,Strd2i  i ,"ST"  ,Vwl ! j) 


) //  Prevocalic  T 

for  i : = 1 to  NStrdlO  do 

( for  j :=  1 to  VwliO  do 

i WtStrs(NStrd|i,"T  #"  ,Vwl ! j ,NStrd ! i ,"TV" ,Vwl i j ) 

) //  Followed  by  StFRIC 

for  i : = 1 to  Strd3!0  do 

{ WtStrsC'T  It"  ,Strd3!  i ,"  = > TS",Strd3!i) 


//  Allows  Glottal 
stressed  vowel. 


tops  at  phrase  beginnings  and  //between  vowels  and  a 
.0  do 

WtStrsC’-  #",S2Vwl !!,"=>  - 0",S2Vwl|i) 


for  i :=  1 to  Vwl'O  do 
for  J :=  1 to  S2Vwl!0  do 


r 1 :=  1 to  S2Vwl!0  do 

WtStrs(  Vwl  | i ,S2Vwl ! j ,Vwl ! i ,"0"  tS2Vwl ! J ) 


//  UW  can 
consonants . 


become  fronted  when  preceded  and  followed  by  coronal  // 


for  i :=  1 to  CrConlO  do 
for  j :=  1 to  PreUYlO  do 

{ WtStrs( PreUY ! j , "UW! 2 #"  ,CrCon ! i , PreUY ! ) , "UY ! 2" ,CrCon i i ) 


//  This  makes  betas  of  the  N and  fl 
for  i 1 to  fjasBIO  do 
( WtStrsi NasB | i , H = >"  , NasD J i , " YN" ) 
WtStrsl  NasP ! i ,"//  fl  =>"  ,NasP|  i ,"YM" ) 


EndWrite ( Out Jfn ) 
finish 
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Appendix  2 


Formats  and  Examples  of  Dictionary  Flies 

In  this  appendix,  we  give  sample  fragments  of  the  various  Hies 
produced  during  dictionary  expansion  (see  Sec.  C).  With  the  exception  of 
the  VERDICT  and  DICTTEXT  files,  all  of  them  are  LISP  symbolic  files  with 
appropriate  dictionary  information  recorded  on  the  property  list  of  each 
entry.  The  format  for  property  lists  on  such  files  is 

[word 

( property  1 value  1 
property2  value  2 

propertyn  valuen  ) ] 

TRAVELDICT . FRAGMENT  is  a fragment  of  the  original  dictionary  file 
before  expansion.  Here,  properties  include  the  syntactic  categories  of 
which  the  word  is  a member  (e.g.,  SPECIAL,  AUX,  V,  PREP,  N,  SPONSOR,  ADJ) 
as  well  as  other  kinds  of  information.  For  example,  the  PHONEMES  property 
has  as  its  value  the  phonetic  pronunciation  or  pronunciations  associated 
with  the  word.  For  each  syntactic  category,  the  value  of  the  property 
indicates  how  the  word  forms  its  inflections  orthographically . For 

example,  an  -S  value  for  the  N property  indicates  that,  as  a noun,  the  word 
forms  its  plural  by  adding  -S.  For  verbs  (V)  only  third  person  singular 
and  past  inflectional  affixes  arp  indicated,  as  all  regular  verbs  form 
their  present  progressive  by  adding  -ing.  The  value  • after  a syntactic 
category  indicates  that  a word  undergoes  the  default  inflection  for  that 
category  (no  inflection  for  most  categories,  but  regular  -er,  -est 
inflection  for  adjectives)  . Inflectional  codes  not  shown  in  the  fragment 
are  S-ED,  ES-ED,  S-D,  IRR,  and  MASS. 

The  value  of  the  PHONEMES  property  is  either  a single  ARPAbet 
"spelling"  for  the  word,  or  a list  of  the  form  ((OR  spl  sp2  ...  spn))  where 
each  spi  is  an  alternative  spelling.  The  general  format  for  a phonetic 
spelling  is: 

((p  fl  f 2 ...  fn)  phi  ph2  ...  phn) 
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whore  £ pronunciation  likelihood  for  the  spelling  (a  number  indicating 
the  likelihood  of  this  particular  pronunciation  of  the  word),  the  fi/s  are 
pronunciation  features  associated  with  this  pronunciation  (e.g.,  REDUCFD 
is  a feature  of  some  pronunciations  of  some  function  words)  , and  the  phi's 
are  the  phonetic  elements  which  make  up  the  particular  pronunciations. 
These  elements  include  "ARPAbet"  phonemes,  the  symbol  • (indicating 
syllable  boundaries),  and  the  symbols  12,  II,  10,  and  !-  (indicating 
decreasing  levels  of  stress).  Both  the  pronunciation  likelihood  and  the 
pronunciation  features  are  optional  and  may  be  omitted. 

INFLECTED. FRAGMENT  is  a fragment  of  the  file  produced  as  a byproduct 
of  the  initial  inflectional  expansion.  Its  format  is  similar  to  that  of 
the  original  dictionary,  except  for  the  addition  of  the  properties  TREE  and 
WORDINDEX  to  each  entry  and  the  creation  of  new  entries  for  regular 
inflected  forms.  The  WORDINDEX  property  assigns  a numerical  ir dex  to  each 
dictionary  entry  to  be  used  for  referring  to  words  by  number  in  interfork 
communication  and  internal  arrays.  The  TREE  property  contains 
pronunciations  in  the  form  required  for  input  to  the  rule  expansion 
program.  This  format  is  similar  to  that  of  the  PHONEMES  property,  except 
that  there  is  always  a pronunciation  likelihood  associated  with  each 
pronunciation  (the  default  value  is  1.0),  and  the  phonetic  spelling  begins 
and  ends  with  a word  boundary  mark  (#).  The  new  entries  for  inflected 
forms  contain  TREE  and  WORDINDEX  properties  of  their  own,  plus  a 
ROOT-INFLECT  property  (which  indicates  the  root  word  of  which  the  new  entry 
is  an  Inflected  form),  the  type  of  inflection,  and  the  syntactic  categories 
of  which  the  new  word  is  a member.  For  example, 

ROOT-INFLECT  (ACCOUNT  -S  N) 

indicates  that  the  entry  is  the  regularly  inflected  (plural)  form  of  the 
noun  ACCOUNT  formed  by  adding  the  ending  -S.  Entries  in  EXPDICT . FRAGMENT 
have  exactly  the  same  format  as  in  INFLECTED  except  that  the  TREE  property 
has  been  replaced  by  the  property  EXPHONES,  which  records  the  different 
phonetic  spellings  that  result  from  the  dictionary  expansion.  The  format 
for  these  phonetic  spellings  is  the  same  as  for  the  TREE  property. 
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The  EXPH0NK3. FRAGMENT  file  gives  an  example  of  the  kinds  of  output 
that  are  generated  for  the  phonologist  to  look  at  to  see  the  effects  of  his 
rules.  It  consists  of  two  sections.  The  first  gives  the  original  phonetic 
spelling,  the  resulting  spellings,  and  the  rules  used  (if  any)  for  each 
entry  in  the  dictionary.  The  second  section  gives  for  each  rule  the  number 
of  times  it  applied  and  a list  of  the  words  to  which  it  applied  (stored  as 
the  value  of  the  atom  RULEHISTORY) . Finally  at  the  end  of  the  file,  a 
summary  of  the  number  of  roots,  words,  and  pronunciations  is  given. 

VFRPICT. FRAGMENT  is  an  example  of  the  file  that  is  created  for  the 
Verification  component.  The  format  of  an  entry  in  this  file  is  (ndx  word 
pronl  pron2  ...  pronn)  where  ndx  is  the  wordindex  of  word  and  each  pronl 
is  a pronunciation  of  the  word  to  be  considered  by  the  Verification 
component.  Most  words  have  only  one  such  pronunciation,  but  occasional 
words  have  two  or  more.  The  format  of  a pronunciation  here  consists  of  a 
sequence  of  the  elements  making  up  its  phonetic  spelling  followed  by  a word 
boundary  mark  ( fl ) . 

DICTTEXT. FRAGMENT  give3  an  example  of  the  dictionary  file  constructed 
for  input  to  the  Lexical  Retrieval  component.  The  format  here  consists  of 
one  pronunciation  per  line,  each  of  which  is  a sequence: 

p <tab>  pfeats  <tab>  pron  <tab>  word  <tab>  categories 

where  j)  is  a pronunciation  likelihood,  pfeats  is  a list  of  pronunciation 
features  (separated  by  spaces)  or  NIL  if  none,  pron  is  the  phonetic 
spelling  of  the  word  (with  stress  marks  incorporated  into  vowels  and 
syllable  boundaries  removed),  and  categories  is  a list  of  the  syntactic 
categories  of  which  the  word  is  a member  (separated  by  spaces).  For 
subsequent  pronunciations  of  the  same  word,  the  common  information  about 
the  word  and  its  syntactic  categories  is  not  repeated. 


<WOODS>TRA VFLDICT . FRAGMENT ; 1 


Tup  ?6-Oct-7b  5:51PM  PAGE  1 


(ART  • 

PHONEMES  ((OR 


[ABOUT 

(PREP  • 
PHONEMES  (AX 
NUM-ADJ  • )] 
[ACCOUNT 

(N  -S 

PHONEMES  (AX 

[ACL 

(PHONEMES  (EY 
SPONSOR  • )] 
[ACOUSTICAL 
(ADJ  * 

PHONEMES  (AX 


(EY  12) 
((REDUCED) 

AH  10)))  )] 


!-  * B AW  !2  T) 


!-  * K AW  12  N T)  )] 

!1  * S IY  !1  * EH  ! 2 L) 

!-  • K UW  \2  * S T IH  !0  • K AX  ! 


<WOODS>INFLECTED. FRAGMENT; 1 Tue  26-Oct-76  3:10PM 


(ART  » 

PHONEMES  ((OR  (EY  !2) 

((REDUCED) 

AH  ! 0) ) ) 

TREE  (OR  ((1.0) 

It  EY  ! 2 It) 

((1.0  REDUCED) 

It  AH  ! 0 #)) 
WORDINDEX  3 )] 

[ABOUT 

(PREP  * 

P..0NEMES  (AX  ! - » B AW  !2  T) 
NUM-ADJ  * 

TREE  ((1.0) 

It  AX  !-  « B AW  !2  T It) 
WORDINDEX  4 )] 

[ACCOUNT 

(N  -S 


PHONEMES  (AX  !-  » K AW  !?  N T) 
TREE  ((1.0) 

It  AX  !-  * K AW  !2  N T //) 
WORDINDEX  5 )J 
[ ACCOUNT-S 

(ROOT-INFLECT  (ACCOUNT  -S  N) 
TREF  ((1.0) 

It  AX  !-  * K AW  !2  N T -S 
WORDINDEX  6 )] 

[ACL 


It) 


v rHUNEMES  ( EY  !1  » S I Y ! 1 * EH  ! 2 L ) 
SPONSOR  * 

TREE  ((1.0) 

fl  ft  EY  ! 1 * S IY  ! 1 » EH  !2  L It) 
WORDINDEX  7 ) ] 


- L)  )] 


PAGE  1 
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<WOODS>EXPDICT . FRAGMENT • 1 


Tue  26-0ct-76  3:13PM  PAGE  1 


[A 

(ART  » 

PHONEMES  ((OR  (EY  !2) 

((REDUCED) 

‘ ' !0))) 

WORDINDEX  3 
EXPHONES  (OR  ((1.0) 

It  EY  12  It) 

( ( .7  REDUCED) 

It  AX  ! - n 
( ( .5  REDUCED) 

It  AH  ! 0 It))  )] 

[ ABOUT 

(PREP  * 

PHONEMES  (AX  !-  • B AW  !2  T) 
n’’M-ADJ  * 

WORDINDFX  U 
EXPHONES  ((1.0) 

It  AX  !-  * B AW  ! 2 T 9)  )] 

[ACCOUNT 

(N  -S 

PHONEMES  (AX  • K AW  !?  M T) 

WORDINDEX  5 
EXPHONES  ((1.0) 

it  AX  !-  • K AW  !2  N T 9)  )] 

[ ACCOUNT-S 

(POOT-INFLECT  (ACCOUNT  -S  N) 

WORDINDEX  6 
EXDHONES  ( ( 1 .0) 

# AX  !-  « K AW  !2  N T S It)  )] 

[ACL 

(PHONEMES  (EY  !1  • 3 IY  !1  « FH  \2  L) 

SPONSOR  • 

WORDINDEX  7 
EXPHONES  ((1.0) 

<7  EY  ! 1 • S IY  ! 1 • EH  !2  L It)  )] 
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<WOODS>EXPHONES. FRAGMENT; 1 Fri  29-Oct-76  5:41PM 


(PHONEMES  ((OR  (EY  !2) 

((REDUCED) 

AH  10))) 

USEDRULES  (22) 

EXPHONES  (OR  ((1.0) 

If  EY  !2  It) 

((  .7  REDUCED) 

# AX  ! - #) 

((  .5  REDUCED) 

It  AH  ! 0 ID)  )] 

[ABOUT 

(PHONEMES  (AX  !-  * B AW  !2  T) 

EXPHONES  ((1.0) 

It  AX  !-  « B AW  !2  T #)  )] 

[ACCOUNT 

(PHONEMES  (AX  !-  » K AW  !2  N T) 

EXPHONES  ((1.0) 

It  AX  !-  * K AW  !2  N T #)  )] 

[ACCOUNT-S 

(USEDRULES  (R-S2) 

EXPHONES  ((1.0) 

It  AX  !-  « K AW  !2  h T S #)  )] 

[ACL 

(PHONEMES  (EY  M * S IY  M « EH  !2  L) 
EXPHONES  ((1.0) 

II  EY  !1  * S IY  !1  * EH  12  L It)  )] 


[ZERO 

(PHONEMES  (Z  IY  !2  * R OW  !0) 
EXPHOL'L'S  ((1.0) 

It  Z IY  !2  * R OW  !0  #)  )] 


PAGE  1 
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<WOODS>FXPHONES . FRAGMFNT ; 1 


Fri  2 q-Oct-76  5:41  PM  PAGE  2 


( LISPXPRINT  (QUOTE  ((V:  ( RULFHISTORY  DICTFILE  IROOTS  #WORDS 

PRONUNCIATIONS  EXPRATIO)))) 


T T) 

( LI3PXPRINT  (QUOTE  (V:  (RULFHISTORY  DICTFILE  #ROOTS  DWORDS 
//PRONUNCIATIONS  EXPRATIO)))  T) 

(DEFINEV 

(RULFHISTORY  ((FRI  23  COMPUTATIONAL  COMPUTE  COMPUTE-ED  COMPUTE-ING 

COMPUTE-S  CONTINUE-ED  CONTINUE-S  FUTURE  HOUSTON  JUNE 
MASSACHUSETTS  MASSACHUSETTS  NEWgYORK  TUESDAY  TUESDAY 
UNIVAC  UNIVAC  USUALLY  USUALLY  USUALLY  USUALLY  UTAH 
UTAH) 

(R-ED1.A  2 LAST-ED  LIST-ED) 

(R-ED1.B  15  ADD-ED  AFFORD-ED  ATTEND-ED  BUDGET-ED 

COMPUTE-ED  CREATE-ED  END-ED  ESTIMATE-V-ED 
NEED-ED  OVER-BUDGET-ED  PRINT-ED  START-ED 
UNDER-BUDGET-ED  VISIT-ED  WANT-ED) 

( R-ED2  2 MARK-ED  STOP-ED) 


(Z57  9 ANY  ANYBODY  ANYBODY  ANYBODY  ANYONE  ANYWHERE  BONNIE 
MANY  MONEY) 

(Z58  6 FORGOTTEN  FORGOTTEN  GOTTEN  TOTAL  TOTAI.-ED  TOTAL-S) 
(Z5A  3 DO  TO  YOU) 

( Z6  H COSELL  NOVEMBER  OTTAWA  PHONOLOGY) 

(Z60  3 ARRANGE  ARRANGE  CHANGE) 

( Z7  10  AMHERST  AMHERST  DOLLAR  DOLLAR-S  FOUR  FOURTH  HER 
RECORD  RFCORD-S  RICHARD) 

(Z8A  1 OUR))) 

( DICTFILE  <SPSYS>TRAVELDICT. ; 20 ) 

(//ROOTS  4 9 1 ) 

(//WORDS  667) 

(//PRONUNCIATIONS  1519) 

(EXPRATIO  2.277361) 

STOP 
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<WOODS>VERDICl  .FRAGMENT ; 1 Tue  26-Oet-76  3:  15PM  PAGE  1 


(1  -PAUSE-  - it  ) 

(2  - S S # Z # AX  !-  Z # ) 

(3  A EY  !2  it  AH  !0  it  ) 

(«  ABOUT  AX  !-  * B AW  !0  T it  ) 

(5  ACCOUNT  AX  !-  * K AW  !2  N T it  ) 

6 ACCOUNT-^  AX  !-  ' K AW  !2  N T S i ) 

(7  ACL  EY  ! 1 • S IY  ! 1 * EH  ! 2 L # ) 

(8  ACOUSTICAL  AX  !-  * K UW  !2  * S T IH  !0  • K AX  !-  L it  ) 

(9  ACOUSTICS  AX  !-  * K UW  !2  * S T IH  10  K S # ) 

(10  ACTUAL  AE  ! 2 K * CH  UW  !0  L it  AE  ! 2 K • CH  UW  i 0 * AX  !-  L # ) 

11  ACTUALLY  AE  ! 2 K * CH  UW  ! 0 * L IY  ! 0 # AE  ! 2 K * CH  UW  !0  • AX  !- 
(12  ADD  AE  !2  D it  ) 

(13  ADD-ED  AE  !2  * D IX  !-  D it  ) 

(itf  ADD-ING  AE  !2  • D IX  !-  NX  # ) 

(15  ADD-S  AE  !2  D Z it  ) 


<WOODS>DICTTEXT. FRAGMENT ; 1 Tue  26-0ct-76  3 : 1 8PM  PAGE  1 


1.0 

NIL 

_ 

-PAUSE- 

SPECIAL 

1.0 

UNVOICED 

-S 

AUX  SPECIAL  V 

1.0 

VOICED 

Z 

1.0 

STRIFRIC 

AX!-  Z 

1.0 

NIL 

EY ! 2 

A 

ART 

.7 

REDUCED 

AX!  - 

.5 

REDUCED 

AH!  0 

1.0 

NIL 

AX!-  B 

AW!  2 T 

ABOUT 

num-adj 

PREP 

.5 

NIL 

AX!-  KA 

AW!  2 T 

ACCOUNT 

N 

1.0 

NIL 

AX!-  KA 

AW!  2 N 

T 

.5 

NIL 

AX!-  KA 

AW! 2 TS  S 

ACCOUNT- 

S 

1.0 

NIL 

AX!-  KA 

AW!  2 N 

TS  S 

1 .0 

NIL 

EY ! 1 S 

I Y ! 1 EH! 2 L 

ACL 

SPONSOR 

.28 

NIL 

AX!-  KA 

UW!  2 S 

T KA  EL! 

_ 

ACOUSTICAL 

.56 

NIL 

AX!-  KA 

UW!  2 S 

ST  IX!- 

KA  EL! - 

• 35 

NIL 

AX!-  KA 

UW!  2 S 

T KA  AX! 

- L 

.7 

NIL 

AX!-  KA 

UW!  2 S 

ST  IX!- 

KA  AX ! - 

L 

.4 

NIL 

AX!-  KA 

UW!  2 S 

ST  IH! 0 

KA  EL! - 

.5 

NIL 

AX!-  KA 

UW!  2 S 

ST  IH! 0 

KA  AX ! - 

L 

.7 

NIL 

AX!-  KA 

UW!  2 S 

ST  IX!- 

K S 

ACOUSTICS 

.5 

NIL 

AX!-  KA 

UW!  2 S 

ST  IH! 0 

K S 
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Appendix  3 - Results  of  Final  Run 


This  appendix  lists,  for  each  sentence  type  and  token  in  the  1?4 
utterance  final  performance  test,  the  results  of  running  on  both 
bIC.DICT-!15  (the  1097  words  accessible  to  BIGGRAM)  and  TRAVELDICT-! 20  (the 
409  words  accessible  to  MIDGRAM)  versions  of  the  HWIM  system. 

The  notation  N1/N2  in  each  entry  denotes  that  N1  theories  were  created 
and  N2  seconds  of  CPU  time  were  consumed. 

Utterances  marked  "Gave  up"  were  terminated  after  150  theories  were 
processed . 
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RMS  102 
BIGDICT: 
TRAVELDICT: 
WAW102 
BIGDICT: 
TRAVELDICT: 


What  is  the  budget  figure? 

Got:  GIVE  ME  TOO  OUR  BUDGET  FIGURE  (1)  3^4/ iqi4  3 
Got:  5HI  TFTE  BTJdTTET  FTGURE  (1)(2)  22/1196 

Correct  23/1795 
Correct  21/1236 


1)  IS  matched  poorly  after  WHAT. 

2)  Semantically  identical  to  the  correct  theory. 


List  all  trips  to  California  this  year. 

JJW106B 

BIGDICT:  Got:  LIST  ALL  TRIP-S  TO  PERU  FOR  HIM  THIS  YEAR  (1)  38/2290 

TRAVELDICT:  Correct  15/1137 

WAW106 

BIGDICT:  Correct  28/2002 

TRAVELDICT:  Correct  29/1772 

(1)  Insufficient  stack  length  in  Lexical  Retrieval. 


What  is  the  registration  fee? 

RMS110D 

BIGDICT:  Got:  GIVE  THE  REGISTRATION  FEE  (1)(2)  12/1169 

TRAVELDICT:  Got:  TSTTE  THE  REGISTRATION  FEE  (1)(2)  12/893 
WAW110C 

BIGDICT:  Correct  1 8/ 1 33^ 

TRAVELDICT:  Correct  18/1106 

(1)  Failed  to  match  IS  after  WHAT. 

(2)  Semantically  identical  to  correct  theory. 


RMS1 19 
BIGDIuT: 
TRAVELDICT: 
WAW1  19B 
BIGDICT: 
TRAVELDICT: 


Who's  going  to  IFIP? 

Correct  99/2655 
Correct  92/2022 

Got:  WHO  IS  GOING  TO  PISA  THEN  (!)  38/2365 
Got:  WHO  IS  GOING  TO  FHafl  TTT96/2086 


(1)  IFIP  matches  poorly  after  TO  due  to  unanticipated  glottal  stop. 


When  is  the  next  ASA  meeting? 

RMS125B 

BIGDICT:  Gave  up  ( 1 ) 150/9659 

TRAVELDICT:  Gave  up  ( 1 ) 150/9337 

WAW 1 25C 

BIGDICT:  Got:  WHEN  IS  THE  NEXT  ASSP  MEETING  (2)  19/1953 

TRAVELDICT:  Correct  11/776 

(1)  Bad  match  for  IS  after  WHEN  and  no  match  for  THE  due  to  bad  lattice. 

(2)  Segment  lattice  had  extra  segments  between  ASA  and  MEETING,  thus 
resulting  in  a worse  score  than  ASSP  MEETING. 


RMS  126 
BIGDICT: 
TRAVELDICT: 
WAW126D 
BIGDICT: 
TRAVELDICT: 


How  much  have  we  already  3pent? 

Got:  HOW  MUCH  IS  BILL  -S  EXPENSE  (1)  138/5309 
Got:  HOW  MUCH  TS  FTIX  2 EXPENSE  (1)  53/2575 

Correct  29/1587 
Correct  29/1138 


(1)  Poor  match  for  HAVE  followed  by  a pcor  verify  score. 
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How  much  is  left? 


f 


JJW137 

BIGDICT: 

TRAVELDICT: 

WAW137D 

BIGDICT: 

TRAVELDICT: 


Got:  WhAT  IS  LEFT  (1)(2)  18/1^33 
Got:  WKT  IS  LEFT  (1)(2)  12/933 

Correct  8/1077 
Correct  8/904 


11  No  correct  seed  words  were  found. 

2)  Semantically  identical  to  the  correct  theory. 


Show  me  a list  of  the  remaining  trips. 

RMS178 

BIGDICT:  Got:  SHOW  ALL  WALLY  -S  FIVE  REMAINING  TRIP-S  (1)  79/3210 

TRAVELDICT:  Got:  SHOW  HX  ~BTIX~ -5~FTVe  REMAINING  TRIP-S  (1)  55/2228 

WAW178  — — 

BIGDICT:  Got:  SHOW  ANY  NEXT  OTHER  REMAINING  TRIP-S  (2)  147/5675 

TRAVELDICT:  Got:  SHOW  HE~A"TT5T~0F  THEIR  REMAINING  TRIP-S  (3)  23/1654 

(11  Failed  to  find  ME  after  SHOW. 

(2)  No  correct  seeds.  SHOW  was  found  to  the  left  of  the  seed  ANY. 

(3)  Fails  to  find  THE  after  OF,  although  it  does  find  THEIR. 


What  is  the  registration  fee  for  the  next  ACL  conference? 

RMS  180 

BIGDICT:  Got:  GIVE  OUR  REGISTRATION  FEE  FOR  THE  NEXT  ACL  CONFERENCE 

( 1 ) 2972tf3B 

TRAVELDICT:  Got:  GIVE  OUR  REGISTRATION  FEE  FOR  THE  NEXT  ACL  CONFERENCE 

(1)  27mi2 

WAW180 

BIGDICT:  Gave  up  (2)  150/4820 

TRAVELDICT:  Got:  WHAT  IS  THE  REGISTRATION  FOR  THE  NEXT  ACL  CONFERENCE 

(2)  (3)  140/3957 


(1)  GIVE  scores  better  than  WHAT,  and  WHAT  never  Rets  seen. 

(2)  FEE  scores  better  than  FOR  after  REGISTRATION  but  cannot  be  followed  by 
FOR  due  to  a 3-to-l  segmentation  error. 

(3)  Semantically  identical  to  the  correct  theory. 


J 

jjwi8:  $ 
BIGDICr: 
TRAVELDICT: 
WAW183 
BIGDICT: 
TRAVELDICT: 


% 

Enter  a trip  for  Bonn.e  Nash-Webber  to  Grenoble. 

Correct  19/1709 
Correct  20/1433 

Gave  up  (1)  150/6058 
Gave  up  (1)  150/5620 


(1)  ENTFR  does  not  score  well. 


JJW185 

BIGDICT: 

TRAVELDICT: 

WAW185 

BIGDICT: 

TRAVELDICT: 


Please  show  me  John  Makhoul's  three  trips  to  Pittsburgh. 

Cot:  SHOW  ME  ALL  OF  THOSE  THREE  TRIP-S  TO  PITTSBURGH  ( 1 ) ( 2) 
41/2536 

Got:  SHOW  ME  ALL  OF  THv,  IE  THREE  TRIP-S  TO  PITTSBURGH  (1) 
33/1699 

Got:  SHOW  ME  JOHN  MAKHOUL  -S  THREE  TRIP-S  TO  PITTSBURGH 
( 1 ) ( 3)  23/1885 

Got:  SHOW  ME  JOHN  MAKHOUL  -S  THREE  TRIP-S  TO  PITTSBURGH 
( 1 ) ( 3 ) 17/1134 


(1)  PLEASE  did  not  score  well  and  received  a poor  verify 
(2  Fails  to  find  JOHN  after  SHOW  ME. 

(3)  Semantically  identical  to  the  correct  theory. 


score . 
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JJW270C 

EIGDICT: 

TRAVELDICT: 

WAW270P 

BIGDICT: 

TRAVELDICT: 


When  did  Craig  go  to  Utah? 

Correct  35/ 19 1 4 
Correct  31/1400 

Correct  57/2720 

Got:  WHAT  IS  THIS  PLANE  FARE  TO  OTTAWA  (2)  20/1212 


JJW272D 

BIGDICT: 

TRAVELDICT: 

WAW272 

BIGDICT: 

TRAVELDICT: 


What  is  the  plane  fare  to  Ottawa? 

Correct  15/143*1 
Correct  15/1475 

Got:  WHAT  IS  THAT  PLANE  FARE  TO  OTTAWA  (1)  27/1826 
Got:  WHAT  IS  TTTTT  PLANE  FARE  TO  OTTAWA  (1)  13/1264 


( 1 ) THAT  PLANE  scores  better  than  THE  PLANE. 

(2)  Spall  MAXSFG  differences  (compared  to  the  BIGDICT  run)  make  THIS  look 
very  good.  The  spanning  theory  grew  from  the  seed  THIS. 


RMS275B 

BIGDICT: 

TRAVELDICT: 

WAW275 

BIGDICT: 

TRAVELDICT: 


Add  a new  budget  item. 

Correct  15/1091 
Correct  14/943 

Correct  85/3154 
Correct  96/3389 


RMS277B 

BIGDICT: 

TRAVELDICT: 

WAW277 

BIGDICT: 

TRAVELDICT: 


The  registration 


fee  is  twenty  dollars. 


Got:  THEIR  REGISTRATION  FEE  IS  20  D0LLAR-S  (1)  33/2074 
Got:  THEIR  REGISTRATION  FEE  IS  20  DOLLAR-S  (l)  24/1095 


Got:  HER  REGISTRATION  COST  FIVE  DOLLAR-S 
Got:  TTEE  REGISTRATION  EH5T  FITE  DOLLAR-S 


27/1730 
44/ 1767 


(1i  THE  is  a poor  seed  and  is  so  short  that  it  can't  be  verified. 

Also,  lattice  contains  an  extra  segment  before  the  R of  REGISTRATION. 

(2)  HER  scores  better  than  THE. 

(3)  FEE  is  a poor  match  next  to  REGISTRATION. 


JJW278C 

BIGDICT: 

TRAVELDICT: 

WAW278 

BIGDICT: 

TRAVELDICT: 


List  the  remaining  untaken  trips. 

Gave  up  (1)  150/4500 
Gave  up  ( 1 ) 150/3684 

Gave  up  (2)  150/4972 
Correct  51/2152 


(1)  THE  matched  poorly  to  the  right  of  LIST. 
k2)  Poor  matches  for  both  REMAINING  and  UNTAKEN. 


JJW279D 
BIGDICT: 
TRAVELDICT : 
WAW279B 
BIGDICT: 
TRAVELDICT: 


Which  trips  were  canceled? 

Gave  up  ( 1 ) 150/4972 
Correct  7^/ 1282 

Got:  ERIC  -S  TRIP-S  WERE  TEN  DOLLAR-S  (2)  141/6141 
Got:  TTTS~TFTP-S  WERE  TEN  rorLAff-!H'2)  85/386O 


(11  Failed  to  notice  a good  match  for  CANCEL-ED  after  WERE,  due  to  an 
inconsistency  in  the  Control/Syntax  interface. 

(2)  WHICH  and  TRIP-S  don't  score  well  and  never  get  seen. 
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RMS280B 

BIGDICT: 

TRAVELDICT: 

WAW280L 

BIGDICT: 

TRAVELDICT: 


How  many  trips  are  there? 

Correct  43/2114 
Correct  39/1557 

Correct  9/1000 
Correct  9/6 22 


RMS283 

BIGDICT: 

TRAVELDICT: 

WAW283B 

BIGDICT: 

TRAVELDICT: 


Cancel  Lyn's  trip  to  the  ASA  meeting. 

Got:  GET  ONLY  ANN  -S  TRIP  TO  THE  ASA  MEETING  (1)  71/3176 
Got:  GFT  5UTHET[rXND  -S  TRIP  TO  THE  ASA  MEETING  (1  ) 102/3611 

Correct  56/2851 
Correct  39 / 1 95b 


(1)  No  correct  seeds,  due  to  po  ' els  in  left  of  lattice. 


Give  me  a list  of  t!  untaken  trips. 

JJW285 

BIGDICT:  Correct  43/253*1 

TRAVELDICT:  Correct  35/1759 

WAW285B 

BIGDICT:  Got:  GIVE  BILL  -S  FIFTY  UNTAKEN  TRIP-S  (1)  43/2240 

TRAVELDICT:  Got:  GIVE  TOGT  FIFTY  UNTAKEN  TRIP-S  (1)  20/1213 

(1)  Failed  to  find  ME  after  GIVE. 


Schedule  a trip  by  train  to  New  York. 

JJW292D 

BIGDICT:  Correct  14/1920 

TRAVELDICT:  Correct  14/1600 

WAW292B 

BIGDICT:  Got:  SCHEDULE  A TRIP  BY  TRAIN  TO  SUNY  (1)  39/2889 

TRAVELDICT:  Got:  SCHEDULE  A TRIP  BY  TRAIN  IN  MKti  ( 1 ) 42/2627 

(1)  NEWgYORK  follows  a poor  match  of  TO,  while  SUNY  follows  a good  one. 


JJW312 

BIGDICT: 

TRAVELDICT: 

WAW312 

BIGDICT: 

TRAVELDICT: 


What  is  the  one-way  air  fare  from  Boston  to  London? 

Correct  41/2241 
Correct  33/1723 

Gave  up  (1)  160/5458 

Got:  WHAT  ARE  THESE  ONE-WAY  AIR  FARE-S  FROM  BOSTON  TO  LONDON 
( 1 ) 126/39TF* 


(1)  Fails  t.o  find  IS  after  WHAT  (triple  merge  in  IS). 


RMS  3 1 5 
BIGDICT: 
TRAVELDICT: 
WAW315 
BIGDICT: 
TRAVELDICT: 


Create  a trip. 

Correct  81/3309 
Correct  53/2056 

Gave  up  ( 1 ) 150/5193 

Got:  ARRANGE  A TRIP  (1)(2)  136/4033 


1)  No  correct  seeds  as  the  lattice  was  poor  around  CREATE. 

2)  Semantically  identical  to  the  correct  theory. 
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RMS  118 
BiGDICT: 
TRAVELDICT: 
WAW31S 
BIGDICT: 
TRAVElDICT: 


Show  his  trip. 

Got:  SHOW  LYN  -S  TRIP  (1)  20/1466 
Got:  SHOW  IYTT  TRIP  (1)  19/1112 

Got:  SHOW  THIS  TRIP  (2)  35/1999 
Got:  SHOW  THT?  TRIP  (2)  .30/1514 


(1)  Failed  to  match  HIS  to  the  right  of  SHOW. 

(2)  THIS  scores  better  than  HIS  to  the  right  of  SHOW. 


RMS320 

BIGDICT: 

TRAVELDICT: 

WAW320 

BiGDICT: 

TRAVELDICT: 


Print  her  fare. 

Gave  up  ( 1 ) 150/4894 
Gave  up  < 1 ) 150/3944 

Gave  up  (2)  150/6359 

Got:  FIND  THEIR  FARE  THERE  (2)  77/3175 


1)  PRINT  does  not  return  as  a correct  seed. 

2)  PRINT  does  not  score  well  due  to  poor  segment  lattice. 


Show  me  her  trips . 

JJW328 

BIGDICT:  Correct  14/1142 

TRAVELDICT:  Correct  17/921 

WAW  ■*23 

BIGDICT:  Got:  SHOW  MADER  -S  TRIP-S  (0  ( 2 ) 88/3717 

TRAVELDICT:  Got:  SHOW  WF  "B!7RTr~-S  TRIP-S  (0(3)  33/1684 

(1)  Lattice  bad  for  SHOW. 

(2)  No  correct  seeds  were  found.  SHOW  was  found  left  of  MADER. 
(O  BERT  matches  much  better  than  HER. 


JJW330P 

BiGDICT: 

TRAVELDICT: 

WAW330 

BIGDICT: 

TRAVELDICT: 


Was  the  trip  expensive? 

Got:  WAS  THAT  TRIP  EXPENSIVE  (1)  34/2140 
Got:  WHEN~TTT~THE  TRIP  TO  SDC  (2)  138/4107 

Got:  WAS  THAT  TRIP  EXPENSIVE  (3)  60/3231 
Got:  WHATTiT  THAT  EXPENSE-S  (2)  (4)  60/3028 


(1)  Density  scoring  made  WAS  THAT  look  better  than  WAS  THE. 

(2)  WAS  and  EXPENSIVE  were  not  allowed  to  start  and  end  due  to  an 
inconsistency  in  the  Control/Syntax  interface. 

(3)  Both  WAS  and  THE  score  poorly  and  also  ret  poor  verify  scores. 

(4)  Grammar  bug  accepted  this  spanning  theory. 


RMS’33 

BIGDICT: 

TRAVELDICT: 

WAW3?3 

BiGDICT: 

TRAVELDICT: 


Do  we  have  a surplus? 

Correct  13/971 

Got : WHO  WILL  LEAVE  THIS  MONTH  ( 1 ) 98/2467 

Control  fork  broke  (2) 

Gave  up  ( 1 ) 150/3605 


(1)  Railed  to  notice  SURPLUS  after  DO  WE  HAVE  A due  to  an  inconsistency 
in  the  Controi/Svntax  interface. 

(2)  Event  garbage  collector  hue. 


je-Kijv 
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RMS331* 
PIGDICT: 
TRAVEL  DICT: 
WAW?39 
BIGDICT: 
TRAVELCICT: 


Why  did  he  visit  SDC? 

Got:  WHY  DID  DEYO  VISIT  SDC  (1)  81/3252 
Correct  69/2283 

Gave  up  (2)  150/5069 
Gave  up  (2)  150/9631 


1)  Failed  to  find  HE  after  DID  due  to  segmentation  error. 

2)  Failed  to  find  WHY  either  as  seed  or  to  the  left  of  DID. 


Who  went  to  IFIP? 

JJW335 

BIGDICT:  Correct  16/1187 

TRAVELDICT:  Correct  19/769 

WAW‘335 

BIGDICT:  Got:  WHO  WENT  TO  INDIA  (1)  35/2033 

TRAVELDICT:  Correct  20/1179 

(1)  IFIP  scored  about  the  same  as  INDIA  (both  poor)  but  was  anchored  off 
a worse  scoring  match  of  TO. 


Are  we  over  the  budget? 

RMS  379 

BIGDICT:  Gave  up  ( 1 ) 150/5093 

TRAVELDICT:  Got:  WERE  WE  OVER  THE  BUDGET  (2)  69/2778 

WAW939 

BIGDICT:  Correct  12/1098 

TRAVELDICT:  Correct  12/990 

(1)  No  correct  seeds  were  found. 

(2)  ARE  WE  scored  worse  than  WERE  WE  due  to  a left  ending  penalty 
associated  with  ARE  (possible  bug). 


Show  me  Bill's  trip  to  Washington. 

RM3795 

BIGDICT.  Got:  SHOW  ONLY  BELL  -S  TRIP  TO  WASHINGTON  (1)(2)  98/3607 
TRAVELDICT:  Correct  6977779 

WAW395B 

BIGDICT:  Got:  SHOW  ME  BELL  -S  TRIP  TO  WASHINGTON  (2)  92/2069 

TRAVELDICT:  Correct  99/17?^ 


1)  ME  scores  poorly  to  right  of  SHOW  and  to  the  left  of  BILL. 

2)  BELL  scores  better  than  BILL.  Verifier  ranks  them  correctly,  but  the 
score  isn't  enough  to  counteract  the  Lexical  Retrieval  scores. 


JJW796 
EIGDICT: 
TrtAVFLDICT : 
WAW796 
BIGDICT: 
TRAVELDICT: 


When  is  the  ACL  Conference? 

Correct  13/1163 
Correct  1 7/808 

Correct  23/1636 
Correct  18/1199 
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RMS  3**7 
BIGDICT: 
TRAVELDICT: 
WAW347 
BIGDICT: 
TRAVELDICT: 


Figure  hin  expenses. 

Got:  FIGURE  THAT  EXPENSF-S  (1)  26/1893 
Got:  FIGURE  7MT  EXPF.NSE-S  (1)  26/ 1 U 31 

Got:  FIGURE  THIS  EXPENSE-S  (1)  25/1603 
Got:  FIGURE  THU  EXPENSE-S  (1)  21/1206 


(1)  THIS  scores  better  than  HIS.  Also,  a grammar  bug  allowed  this 
theory  to  be  accepted;  otherwise,  would  have  accepted 
FIGURE  THIS  EXPENSE. 


RMS348 

BIGDICT: 

TRAVELDICT: 

WAW3U8 

BIGDICT: 

TRAVELDICT: 


Display  all  taken  trips. 

Correct  111/11267 
Correct  58/2228 

Correct  15/13^4 
Correct  1*1/917 


RMS  311 9 
BIGDICT: 
TRAVELDICT: 
WAW3H9 
BIGDICT: 
TRAVELDICT: 


I will  fly  to  San  Diego. 

Got:  WHO  WILL  FLY  TO  SAN0DIEGO  SOON  (1)(2)  33/1805 
Got:  WU  WILL  FLY  TO  SAN0DIEGO  TTT“38/ 1 69M 

Correct  1*1/1176 
Correct  1*1/873 


1)  WHO  scores  better  than  I due  to  non-optional  HH  in  segment  lattice. 

2)  Bad  segment  lattice  requires  SAN0DIEGO  to  buy  an  ending  penalty,  or 
else  accept  another  word  on  the  right. 


RMS  350 
BIGDICT: 
TRAVELDICT: 
WAW350 
BIGDICT: 
TRAVELDICT: 


When  is  Chip's  trip? 

Correct  18/1298 
Correct  14/981 

Correct  23/1374 
Correct  19/966 


JJW351 

BIGDICT: 

TRAVELDICT: 

WAW351 

BIGDICT: 

TRAVELDICT: 


List  all  fares  to  Chicago. 

Correct  31/19*11 
Correct  29/1541 

Correct  25/1718 
Correct  20/1262 


JJW352 

BIGDICT: 

TRAVELDICT: 

WAW352 

BIGDICT: 

TRAVELDICT: 


How  much  did  Geoff  spend? 

Got:  HOW  MUCH  DID  CHIP  SPEND  (1)  42/2125 
Got:  HOW  MUCH  DID  CTTTF  SPEND  (1)  25/1216 

Got:  HOW  MUCH  DID  JIM  SPEND  (1)  20/1546 
Got:  HOW  MUCH  DID  7KCK  SPEND  (1)  16/1121 


( 1 ) Fails  to 


find  GEOFF  at  all,  due  to  poor  labeling 


in  lattice. 
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RMS353 

BIGDICT: 

TRAVELDICT: 

WAW3^3 

BlGDICT: 

TRAVELDICT: 


List  all  remaining  trips. 

Got:  LIST  ALL  THE  EIGHTY  TRIP-S  (1)  18/1451 
Got:  LIST  ALL  THE  KInTTRIP-S  (1)  15/1175 

Correct  10/1025 
Correct  10/770 


(1)  Fails  to  find  REMAINING  due  to  bad  segment  lattice. 


RMS354 

BIGDICT: 

TRAVELDICT: 

WAW354 

BIGDICT: 

TRAVELDICT: 


Add  a trip  to  France. 


Correct  20/1412 
Correct  30/1441 


Got:  ADD  IT  TO  BERT  -S  TICKET-S  (1)  143/5695 
Got:  ADD  IT  TO  TOFT RlCm  TO5T-S  ( 1)  62/2634 


(1)  TRIP  and  TO  scored  poorly. 


JJW355 

BIGDICT: 

TRAVELDICT: 

WAW355 

BIGDICT: 

TRAVELDICT: 


How  many  trips  has  Rich  taken? 

Gave  up  (1)(2)  150/4653 

Got:  HOW  MANY  TRIP-S  HAS  BERT  TAKEN  (2)  74/2747 

Correct  20/1332 
Correct  1 8/996 


v 1 1 MANY  scores  poorly  causing  the  searching  of  many  other  theories. 
(2)  BERT  matches  much  better  than  RICH  right  of  HAS. 


RMS356 

BIGDICT: 

TRAVELDICT: 

WAW356 

BIGDICT: 

TRAVELDICT: 


Who  went  to  Los  Angeles  this  year? 

Correct  31/2265 
Correct  20/1508 

Correct  14/1345 
Correct  13/996 


RMS  357 
BIGDICT: 
TRAVELDICT: 
WAW357 
BIGDICT- 
TRAVELDICT: 


When  is  Jack  going  to  San  Francisco? 

Correct  20/1643 
Correct  ?"/1385 

Got:  WHEN  IS  ELLIOT  GOING  TO  SANgFRANCISCC  (1)  58/2737 
Got:  WHEN  IS  THAT  'TRIP  FOR  LYN  TO  SAN0FRANCISCO  (1)  147/5007 


(1)  Fails  to  find  JACK  due  to  segment  lattice  problems. 


JJW358 
BIGDICT: 
TRAVEl DICT: 
WAW358 
BIGDICT: 
TRAVELDICT: 


What  is  the  roued-trin  fare  to  Chicago? 

Got:  WHAT  IS  GLENN  ^S  AIR  FARE  TO  CHICAGO  (l)  54/2644 
Correct  33/1676 

Correct  19/1499 
Correct  27/1463 


(1)  THE  scores  poorly  after  WHAT  IS  due  to  poor  segment  lattice  there. 
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Who  went  to  Santa  Barbara  in  August? 


JJW359 

BIGDICT: 

TRAVELDICT: 

WAW3S9 

BIGDICT: 

TRAVELDICT: 


Correct  52/2200 
Correct  ft  1/1692 

Correct  17/1253 
Correct  12/861 


JJW360 

BIGDICT: 

TRAVELDICT: 

WAW360 

BIGDICT: 

TRAVELDICT: 


Does  the  speech  budget  have  a surplus? 

Gave  up  (1)  150/5231 
Gave  up  (2)  151/5669 

Gave  up  m 150/5020 
Gave  up  (3)  150/3752 


(1)  Gets  as  far  as  DOES  THE  SPEECH  BUDGET  HAVE  A but  is  swamped  with  bogus 
w project  names” • 

(2)  Failed  to  notice  SURPLUS  after  DOES  THE  SPEECH  BUDGET  HAVE  A due  to  an 
inconsistency  in  the  Control/Syntax  interface. 

(3)  Fails  to  match  HAVE  after  DOES  THE  SPEECH  BUDGET  due  to  poor  segment 
lattice . 


Why  did  Jerry  go  to  Univac? 

JJW361 

BIGDICT:  Got:  WHY  DID  FEEHRER  ATTEND  AI#LAB  (1)  95/4295 

TRAVELDICT:  Gave  up  ( 1 ) iWETTE 

WAW36 1 

BIGDICT:  Gave  up  (1)  150/550 2 

TRAVELDICT:  Gave  up  ( 1 ) 1 50/^390 

(1}  Fails  to  find  JERRY  due  to  missing  word  boundary  rule  (D  # J =>  J). 


How  much  did  we  spend  in  December? 

JJWR62 

BIGDICT:  Got:  ANN  WENT  TO  NICE  TILL  DECEMbER  (1)  61/2847 

TRAVELDICT:  Got:  WTO  WENT  TO  METlETTN  DECEMBER  ( 1 ) 32/1542 

WAWR62 

BIGDICT:  Got:  HOW  MUCH  DID  WE  SPEND  IN  THE  SUMMER  (2)  55/2596 

TRAVELDICT:  Got:  HOW  MUCH  DID  WE  SPEND  IN  THE  SUflEEP  (2)  251443 

(1)  HOW  scored  poorly  and  never  got  to  the  top  of  the  queue. 

(2)  DECEMBER  scores  better  than  THE  but  not  as  good  as  TH^  SUMMER. 
Verifier  ranks  them  correctly,  but  not  enough  to  offset  the  other 
scores . 


Schedule  a trip  for  Jack  Klovstad  to  SDC. 

JJW365 

BIGDICT:  Gave  up  ( 1 ) 150/6037 

TRAVELDICT:  Cor-ect  42/2273 

WAW365 

BIGDICT:  Gave  up  (2)  150/5936 

TRAVELDICT:  Gave  up  (?)  150/4736 

(1)  Fails  to  find  KLOVSTAD  after  JACK. 

(2)  Fails  to  find  JACK  due  to  segmentation  error  (3  segments  for  vowel). 


JJW366 

BIGDICT: 

TRAVELDICT: 

WAW366 

BIGDICT: 

TP  .VELDICT: 


How  much  money  is  in  the  current  budget? 

Got:  WHAT  IS  CONNIE  -S  ENTIRE  BUDGET  (1)  75/3407 
Gave  up  OT- l5o/4529 

Correct  18/1644 
Correct  15/1105 


(1)  HOW  13  found  as  a seed,  but  its  abysmal  score  means  it  never  gets  seen. 
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JJVH67 

BlGDICT: 

TRAVELDICT: 

WAW367 

BlGDICT: 

TRAVELDICT: 


The  robot  budget  has  7 K. 

Gave  up  (1)  1 BO/5743 
Gave  up  (1)(2)  150/4300 

Correct  53/2750 
Correct  31/1542 


(1)  Gets  as  far  as  THE  ROBOT  BUDGET  HAS  SEVEN  but  SEVEN  buys  a very  large 
compatibility  penalty  (due  to  a matcher  bug). 

(2)  Falls  to  match  K at  the  end  — vowel  labeled  as  nasal. 


The  trip  number  is  4 3 7 6. 

RMS369 

BlGDICT:  Got:  THE  TRIP  NUMBER  IS  5376  (1)  118/5613 

TRAVELDICT:  Gave  up  (2)  150/6063 

WAW369 

BlGDICT:  Correct  72/3648 

TRAVELDICT:  Correct  66/2794 

(1)  FOUR  scores  about  the  3ame  as  FIVE  but  extra  fricative  before  THREE 
makes  FIVE  THREE  look  better  than  FOUR  THREE. 

(2)  THE  scored  poorly  and  was  never  seen  again  because  of  other 
better-scoring  theories. 


JJW370 

BlGDICT: 

TRAVELDICT: 

WAW370 

BlGDICT: 

TRAVELDICT: 


What  trips  remain  in  the  speech  budget? 

Gave  up  (1)  150/61 38 
Gave  up  (1)  150/5446 

Correct  14/1436 
Correct  14/976 


(1)  Failed  to  find  REMAIN  after  WHAT  TRIP-S  due  to  a segmentation  problem. 


JJW371 

BlGDICT: 

TRAVELDICT: 

WAW371 

BlGDICT: 

TRAVELDICT: 


My  trip  to  Mexico  cost  eight  hundred  dollars. 

Got:  A TRIP  TO  MEXICO  COST  A HUNDRED  FOUR  BUCKS  ( 1 ) (2) (3) 
34/2076 

Gave  up  ( 1 ) ( 2) ( 3)  150/4656 

Got:  MY  TRIP  TO  MEXICO  COST  A HUNDRED  DOLLAR-S  (2)  142/4720 
Got:  MY  TRIP  TO  MEXICO  COST  J HUNDRED  DOLLAR-S  (2)  47/1886 


( 1 ) MY  scores  very  poorly  since  there  was  no  segment  for  M. 

(2)  No  T in  EIGHT. 

(3)  DOLLAR  matches  after  HUNDRED  but  not  DOLLAR-S. 


RMS372 
BlGDICT: 
TRAVELP . CT : 
WAW372 
BlGDICT: 
TRAVELDICT: 


Please  give  me  the  actual  cost  of  trip  number  6873- 

Gave  up  (1)  150/6730 
Syntax  fork  broke  (1)(2) 

Gave  up  (1)  150/6239 

Got:  WHERE  IS  THE  D.C.  WORKSHOP  ON  THE  EIGHTH  OF  NOVEMBER 
SIXTr3ETCNn-lTTl77TCir^'~^~ 
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(1)  No  correct  seeds. 

(2)  Grammar/dictionary  bug. 


RMS 37 3 
BIGDICT: 
TRAVELDICT : 
WAW373 
BIGDICT: 
TRAVELDICT: 


What  is  the  cost  of  my  trip? 


Got:  WHY  DID  DICK  GO  TO  IFIP  (1)  70/3202 

Got:  T^A^GTTETTTTElircCF5T~OF  THE  TRIP  (2)  84/305? 


Correct  27/1854 
Correct  24/1486 


(1)  IS  scores  poorly  and  WHAT  IS  never  gets  to  the  top  of  the  queue. 

(2)  IS  scores  poorly  but  WHAT  IS  eventually  reaches  top  of  queue,  finds 
THE  and  is  effectively  killed  by  a large  compatibility  penalty 
(matcher  bug) . 


How  much  is  in  the  speech  understanding  budget? 

JJW374 

BIGDICT:  Gave  up  ( 1 ) 150/6192 

TRAVELDICT:  Gave  up  ( 1 ) 150/5222 

WAW374 

BIGDICT:  Correct  40/2148 

TRAVELDICT:  Correct  40/ 1555 

(1)  Quickly  gets  to  HOW  MUCH  IS  IN  THE  SPEECH  UNDERSTANDING  and  fails  to 
find  BUDGET,  due  to  poor  segmentation  and  labeling 


JJW375 

BIGDICT: 

TRAVELDICT: 

WAW375 

BIGDICT: 

TRAVELDICT: 


Where  is  the  next  ASA  meeting? 

Correct  14/1343 
Correct  l4/8b9 

Correct  19/1 593 
Correct  15/1 078 


Lyn's  trip  to  France  cost  six  hundred  dollars. 

JJW376  , x 

BIGDICT:  Got:  LYN  -S  TRIP  TO  FRANCE  COSTS  SIX  HUNDRED  DOLLAR-S  (1) 

43/2654 

TRAVELDICT:  Got:  LYN  -S  TRIP  TO  FRANCE  COSTS  SIX  HUNDRED  DOLLAR-S  (1) 

43/2098 

WAW376 

BIGDICT:  Got:  GLENN  -S  TRIP  TO  FRANCE  COSTS  SIX  HUNDRED  DOLLAR-S  (1) 

(2)  3l7T$7i 

TRAVELDICT:  Got:  LYN  -S  TRIP  TO  FRANCE  COSTS  SIX  HUNDRED  DOLLAR-S  (1) 

34/1920 

(1)  The  competing  events  were  very  close  in  score  although  COSTS  SIX  scored 
slightly  better  than  COST  SIX. 

(2)  Bad  vowel  label  in  LYN  scored  worse  than  the  initial  G in  GLENN  against 
a pause . 


Show  me  all  the  trips  to  El  Paso. 

RMS377 

BIGDICT:  Got:  SHOW  ALL  LEAVITT  -S  TRIP-S  TO  EL0PASO  (1H21  47/2217 

TRAVELDICT:  Gave  up  ( lTT50/^M 

WAW">77 

BIGDICT:  Correct  15/1218 

TRAVELDICT:  Correct  16/1077 

(1)  No  correct  seeds. 

(2)  SHOW  has  an  L inserted  between  SH  and  OW  and  ALL  is  matched  at  the 
wrong  place  due  to  a very  deep  lattice. 
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RMS378 

BIGDICT: 

TRAVELDICT: 

WAW378 

BIGDICT: 

TRAVELDICT: 


Enter  a trip  to  Grenoble. 

Got:  GET  THEIR  TRIP  TO  GRENOBLE  (1)  59/3024 
Got:  jjgT  THKIH  TRIP  TO  GRENOBLE  (1)  56/2459 

Correct  11/1120 
Correct  11/848 


(1)  ENTER  scores  poorly  as  a seed  and  is  effectively  killed  when  a poorly 
scoring  A is  found  on  its  right. 


Appendix  H 


Performance  Results  For  Strategy  Variations 

This  appendix  presents  the  results  for  the  strategy  variation 

experiments  described  in  Section  F.  The  set  of  10  utterances  was  selected 
at  random  from  the  larger  test  set  of  12H.  Note  that  for  these 
experiments,  the  "give  up"  threshold  was  set  to  100  theories  r'ther  than 
the  ISO  used  in  the  final  performance  runs. 

The  notation  N1/N2  denotes  that  N1  theories  were  created  and  N2 
seconds  of  CPU  time  were  consumed. 
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Experiment  1. 

Scoring  variations  using  the  Left  Hybrid  strategy  with  No  Verification. 


LHQNV : using  Quality  scoring 

LHQDNV : using  Quality  Density  scoring 

LHSNV:  using  Shortfall  scoring 

LHSDNV : using  Shortfall  Density  scoring 


LHQNV 

LHQDNV 

msNy 

LHSDNV 

WAW346 

CORRECT 

33/1790 

CORRECT 

36/1724 

Gave  up 
100/4130 

CORRECT 

29/1656 

WAW183 

Gave  up 
100/3^31 

Gave  up 
100/3785 

Gave  up 
100/4933 

Gave  up 
100/4064 

JJW323 

CORRECT 

18/1147 

CORRECT 

35/1506 

Gave  up 
100/3886 

Correct 

30/1511 

WAW292B 

Incorrect( 1 ) 
24/1757 

Incorrect( 1 ) 
92/4563 

Gave  up 
100/5973 

Incorrect(  1 ) 
32/2713 

WAW351 

CORRECT 

52/1700 

CORRECT 

85/2722 

Gave  up 
100/3577 

CORRECT 

67/2407 

WAW119B 

Gave  up 
100/2432 

Gave  up 
100/2685 

Gave  up 
100/4243 

Incorrect(2) 

92/2845 

WAW280D 

CORRECT 

7/397 

Gave  up 
100/2685 

Gave  up 
100/3686 

CORRECT 

10/864 

WAW348 

CORRECT 

31/1469 

CORRECT 

70/2579 

Gave  up 
100/3777 

CORRECT 

32/1612 

WAW370 

CORRECT 

9/844 

CORRECT 

42/1519 

Gave  up 
100/4199 

CORRECT 

15/1393 

JJW365 

Gave  up 
100/2564 

Gave  up 

IOO/3583 

Gave  up 
100/4701 

Gave  up 
100/3341 

(1)  SCHEDULE  A TRIP  BY  TRAIN  TO  INDIA 

(2)  WHO  IS  GOING  TO  PISA  THEN 


Experiment  2. 

Scoring  variations  using  the  general  (middle-out)  strategy 

with  No  Verification. 


QNV:  using  Quality  scoring 

QDNV : using  Quality  Density  scoring 

SNV:  using  Shortfall  scoring 

SDNV:  using  Shortfall  Density  scoring 


QNV 

QDNV 

SNV 

SDNV 

WAW3*46 

CORRECT 
1 1/5^9 

Gave  up 
100/3252 

Gave  up 
100/4091 

Gave  up 

IOO/3255 

WAW183 

Syntax  fork 
broke  ( 1 ) 

Syntax  fork 
broke  (1) 

Gave  up 
100/4431 

Gave  up 
100/3672 

JJW323 

CORRECT 

7/1423 

CORRECT 

514/21146 

Gave  up 
10C/3966 

CORRECT 

23/1263 

WAW292B 

Incorrect(2) 

22/1173 

Gave  up 
100/4410 

Gave  up 
100/4600 

Gave  up 
100/3866 

WAW351 

Gave  up 
100/33>47 

Gave  up 
100/3551 

Gave  up 
100/3995 

CORRECT 

79/2777 

WAW119B 

Gave  up 
100/14219 

Gave  up 
100/3998 

Gave  up 
100/4810 

Gave  up 
100/3998 

WAW280D 

CORRECT 

7/598 

CORRECT 

19/1850 

Syntax  fork 
broke  (3) 

CORRECT 

32/1502 

WAW348 

Incorrect(i4) 

88/2827 

Gave  up 
100/3494 

Gave  up 
100/4051 

CORRECT 

80/2924 

WAW370 

CORRECT 

9/570 

CORRECT 

49/2267 

Gave  up 
100/4174 

CORRECT 

42/1668 

JJW365 

Gave  up 
100/2279 

Gave  up 
100/41 1(7 

Control  fork 
broke  (5) 

Gave  up 
100/3904 

(1)  Grammar /dictionary  bug 

(2)  SCHEDULE  A TRIP  BY  TRAIN  TO  INDIA 

(3)  Grammar  bug 

(14)  DISPLAY  TEN  TRIP-S 
(5)  Unknown  bug  in  Control. 
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Experiment  3* 

Effect  of  heuristics  on  the  general  (middle-out)  strategy, 
using  Shortfall  Density  scoring  with  No  Verification. 


SD+0:  using  no  additional  heuristics 

SD+C:  using  Collision  events 

SD+G:  using  Ghosts 

SD+GD:  using  Ghosts  and  CHOOSEDIR 

SD+GDC:  using  Ghosts,  CHOOSEDIR,  and  Collisions  (same  as  SDNV) 


SD+0 

SD+C 

SD+£ 

SD+GD 

SD+GDC 

WAW346 

Gave  up 
100/3852 

Gave  up 
100/3895 

Gave  up 
100/3864 

Gave  up 
100/3176 

Gave  up 
100/3255 

WAW183 

Syntax  fork 
broke  ( 1 ) 

Syntax  fork 
broke  ( 1 ) 

Syntax  fork 
broke  (l) 

Gave  up 
100/3654 

Gave  up 
100/3672 

JJW323 

CORRECT 

84/3016 

CORRECT 

29/1753 

CORRECT 

77/2880 

CORRECT 

64/2645 

CORRECT 

23/1263 

WAW2925 

Syntax  fork 
broke  (2) 

Syntax  fork 
broke  (2) 

Gave  up 
100/4421 

Gave  up 
100/4220 

Gave  up 
100/3866 

WAW351 

Gave  up 
100/3611 

Gave  up 
100/3454 

Gave  up 
100/3098 

Gave  up 
100/3456 

CORRECT 

79/2777 

WAW119B 

Gave  up 
100/4322 

Gave  up 
100/4335 

Gave  up 
100/4464 

Gave  up 
100/4044 

Gave  up 
100/3998 

WAW280D 

CORRECT 

39/1862 

CORRECT 

44/1876 

CORRECT 

32/1570 

CORRECT 

31/1790 

CORRECT 

32/1502 

WAW348 

Gave  up 
100/3442 

Gave  up 
100/3312 

Gave  up 
100/3202 

CORRECT 

79/2669 

CORRECT 

80/2984 

WAW370 

CORFECT 

39/1853 

CORRECT 

48/1924 

CORRECT 

36/1792 

CORRECT 

35/1709 

CORRECT 

41/1668 

JJW365 

Gave  up 
100/3995 

Gave  up 
100/3900 

Gave  up 
100/3767 

Gave  up 
100/3893 

Gave  up 
100/3904 

(1)  Grammar/dictionary  bug 

(2)  Grammar  bug 
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Experiment  4. 

Effect  of  Verification  strategies,  using  the  Left-Hybrid  strategy 
with  Shortfall  Density  scoring. 


LHSDNV : using  No  Verification 

LHSDVD:  using  Verify-at-Do 

LHSDVP:  using  Verify-at-Pick  (the  12  October  HWIM  strategy) 


LHSDNV 

LHSDVD 

LHSDVP 

WAW346 

CORRECT 

CORRECT 

CORRECT 

29/1656 

29/1826 

23/1636 

WAW183 

Gave  up 

Gave  up 

Gave  up 

100/4064 

100/4301 

150/6058 

JJW323 

CORRECT 

CORRECT 

CORRECT 

30/1511 

19/1191 

14/1142 

WAW292B 

Incorrect  (1) 

Incorrect  (2) 

Incorrect  (2) 

32/2713 

47/3378 

39/2889 

WAW351 

CORRECT 

CORRECT 

CORRECT 

67/2407 

46/1997 

25/1718 

WAW119B 

Incorrect  (3) 

Incorrect  (3) 

Incorrect  (3) 

92/2845 

47/2256 

38/2365 

WAW280D 

CORRECT 

CORRECT 

CORRECT 

10/864 

9/1064 

9/1000 

WAW348 

CORRECT 

CORRECT 

CORRECT 

32/1612 

25/1653 

15/1344 

WAW370 

COFRFOT 

CORRECT 

CORRECT 

15/1393 

13/1405 

14/1436 

JJW365 

Gave  up 

Gave  up 

Gave  up 

100/3341 

100/3937 

150/6037 

(1)  SCHEDULE  A TRIP 
(?)  SCHEDULE  A TRIP 
(3)  WHO  IS  GOING  TO 


BY  TRAIN  TO  INDIA 
BY  TRAIN  TO  SUNY 
PISA  THEN 


Appendix  5. 


The  following  pages  contain  an  alphabetical  listing  of  the  base 
forms  in  BIGDICT,  the  larger  of  the  two  dictionaries  used  in  HWIM's 
travel  budget  management  domain.  Those  items  also  in  TRAVELDICT,  the 
smaller  dictionary,  are  marked  with  an  asterisk. 
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♦-PAUSE- 

*-S 

♦A 

a. i.n. 

ABORT 

♦ABOUT 

ACAPULCO 

♦ACCOUNT 

ACCURATE 

ACCURATELY 

♦ACL 

ACM 

♦ACOUSTICAL 
♦A CO IJST  ICS 
♦ACTUAL 
♦ACTUALLY 
♦ADD 

♦ADDITIONAL 
ADELAIDE 
♦AFPORD 
AFIPS 
♦AFT ER 
♦AGAIN 
AGR  rcUL'r  JR  E 
♦A  I 

AlrOL  AU 
AIELLO 
AIGHES 
♦AIR 

♦AIRPLAN  F 
ALABAMA 
ALAMOGORDO 
A LASKA 
ALBANY 
A LBUQUERQUE 
ALICE 
♦A  LL 
ALLEN 
♦ALLOW 
♦ALMOST 
♦ALREADY 
♦ALSO 
♦AM 

AMARILLO 
♦AMHERST 
♦AMOUNT 
♦A  MSDEN 
AMSTERDAM 
♦A  N 

AN  A HEIM 


AN ALYS IS 
A NCR  OR  AGE 
♦AND 

ANDERSON 

ANN 

A NNiAP  BOR 
ANOTHER 
ANSWER 
♦ANY 

♦ANYBODY 
♦ANYONE 
ANYTHING 
♦ANYWHERE 
APPLETON 
♦APRIL 
♦ARE 
AR  GONN  E 
ARIZONA 
ARK  ANS AS 
♦ARPA 
♦ARR  ANG  E 
ARTHUR 
♦ASA 
A. SIS 
ASPEN 

♦ASSOCI ATION 
ASS  P 
♦ASSUME 
ATHENS 
ATLANTA 
ATL  ANT  TG3CITY 
♦ATTEND 
AUCKLAND 
♦AUGUST 
AUGUST  A 
AURIGEMHA 
♦AUSTIN 
AUSTRALIA 
AUSTRIA 
♦AUTO 

♦AVAILABLE 
♦A  V EF  AG F 
♦AWAY 
♦PAGIIDAD 
BALTIMORE 
P A NGKOK 
BAR  BAR  A 
BARON 
BARRY 
♦BATES 


♦PBN 

♦RE 

BECKER 

♦BEEN 

♦BEFORE 

♦BEGIN 

♦BEIRUT 

BELGIUM 

BELL 

BBLL^L ABS 
BERKELEY 
BERLIN 
BERLINER 
BERMUDA 
♦BERT 
♦BETWEEN 
♦BILL 
BILLY 

BINGHAMTON 

BIOPSY 

BLADDER 

BLOOMINGTON 

BOB 

BO  BROW 

BOCAdRATON 

BOISE 

BOLOGNA 

BONN 

BONNEVILLE 
♦BONNIE 
BOOK 
♦BOSTON 
♦BOTH 
BOULDER 
BOWLINGJGREEN 
BR  ACHH  AN 
BRAZIL 
BRAZILIA 
♦BREAKDOWN 
BREN  DA 

BRIGHA  MdYQU  NG 
RROWN 
♦BRUCE 
PUCHAR  EST 
BUCKS 
BUDAPEST 
♦BUDGE? 

♦BU  DGFTTED 
BUDIANSKY 
BUFFALO 
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BURCHP  I EL 
BURLINGTON 
BURTON 
♦BUS 
♦BY 
CM  RO 
CALCUL  ATE 
CALCUTTA 
♦CALENDAR 
CALOARV 
♦CALIFORNIA 
CALLSVA 
CALTECH 
CALVIN 
CAMBRIDGE 
♦CAN 
♦CANADA 
♦CANCEL 
CAP  EdTOW N 
♦CAR 

CAR  BON  ELL 
CARL 

♦CARNEGIE 
♦CARNEGIE-MELLON 
CAROL 
CATALI  NA 
CATHY 
♦CENT 
CENTER 
CHAMPA  IGN 
♦CHANGE 
♦CHANGE 
CHARLEN F 
CHARLES 
CHARLOTTE 
CHATTANOOGA 
CHERRY  D HILL 
CHEST 
CHEY  ENNE 
♦CHICAGO 
CHILE 
♦CHIP 
CHI  PM  AN 
CHPI  STCHIJRCH 
CINCIN  ATTI 
♦CITY 
CL  A REN  CP 
CLEARWATER 
CL  EM  FNTS 
CLEVELAND 


CLI  NIC 
♦CM  !I 
♦COAST 
CODE 

♦COLA  RUSSO 
COLL  EG  ,'*PARK 
COLLINS 
COLORADO 
COLOPADOSSPRINGS 
COLUMBUS 
COMBS 
COMMERCE 
COMPCON 
♦COMPRESSION 
♦COMPUTATIONAL 
♦COMPUTE 
COMPUTER 
♦ CON  PER  ENCE 
CONNECTICUT 
CONNIE 
CONSULTING 
♦CONTAIN 
♦CONTINUE 
♦CONTRACT 
♦COOK 

COP  ENH  HAGEN 
CORNELL 

CORPUS9CHRTSTI 
CORRECT 
CORK  EL  ATE 
♦COSELL 
♦COST 

♦COST-PAST 
♦COSTS 
COTCO 
♦COULD 
♦COUNTRY 
♦CRAIG 
♦CREATE 
*CR  ETE 
CUNY 
♦CURRENT 
CZECHOSLOVAKIA 
♦n.  C. 

DALLAS 

DAN 

DARGAN 
DARTMOUTH 
DAPYLE 
DAV  E 


DAV  ID 
♦DAY 
DAYTON 
DEiKLEER 
DEiiPFS  A 
DECATUR 
♦DEC  EMBER 
DECISION-MAKING 
♦DEFICIT 
DELAWARE 
DELETE 
DELHI 
DENMARK 
♦DENNIS 
DEN  V ER 
DEPART 
DETROIT 
DEVELOPMENT 
DEYO 
DICK 
♦DID 

DIFFERENT 
DISCRIMINATION 
DI SKIN 
♦DISPLAY 
DISREGARD 
DIVIDE 
*DO 
DODDS 
♦DOES 
♦DOLLAR 
♦DOMESTIC 
DON 

♦DON-T 
♦ DON  E 
DOUG 
DR  PX  PL 
DUBROVNIK 
DU  BH 00 E 
DULUTH 
DUNCAN 

DUNDEE 

♦DURING 

♦EACH 

♦FAHIY 

♦EAST 

EAST^L AN  SING 
EDINBURGH 
EDMONTON 
EGYPT 


i 


♦FlGii  t 
♦FI  Girt  EN 
♦EIGHTF  i.  MT|) 
♦FTGirH 
♦EIGHTY 
♦EITHFR 
♦FLaPASO 
♦EL  EV  EN 
♦ELEVENTH 
FLL  IOTT 
ELSIE 
EMOTIONAL 
♦END 

♦ENGL  AN  D 
♦ENOUGH 
♦ENTER 
ENTIRE 
ERIC 
ERIE 

♦ESTIMAT  E-N 
♦ESTIMATE-V 
EUGENE 
EUROPE 
♦EUROPEAN 
♦EVERY 
FV FHYUODY 
♦EVERYONE 
♦EV  EHYT  HING 
EXACT 
♦EXACTLY 
♦EXPENSE 
♦EXPENS  IVE 
FACTOR 
FAIRBANKS 
♦FALL 
♦RAF  R 
♦FEU PDA  RY 
♦FEE 
FEEHPE 9 
F El.  DM  AN 
FEU9ZEIG 
♦FI  FTEFN 
♦FIFTEENTH 
♦FIFTH 
♦FIFTY 
♦PIGURF 
♦FINAL 
♦FIN  D 
♦FINDS 
♦FIRST 


♦FISCAL 
♦FI  VR 
FLAGSTAFF 
♦FLEU 
♦FL IRS 
FLI  NT 
FLORENCE 
♦FLORIDA 
♦FLOWN 
♦FLY 
♦FOR 

♦FOREIG  N 
♦FORGET 
♦FORGETS 
♦FORGOT 
♦FORGOTTEN 
FORTaL  AUDERDALE 
FORTaWA YNE 
FORTawORTH 
PORTMA  NN 
♦FORTY 
♦FOUND 
♦FOUR 
♦FOURTEEN 
♦ FOURTEENTH 
♦FOURTH 
♦FRANCE 
FRANK 
FRANKFORT 
FREEMA  N 
♦FRIDAY 
♦FROM 
♦FUTURE 
GAIL 

GAINESVILLE 
GALLUP 
GALVESTON 
♦GAVE 
GENEVA 
♦GEOFF 
♦GEOFFR  EY 
GEORGE 
GEORGI A 
GERMANY 
♦GET 
♦GETS 
♦GIVE 
♦GIVEN 
♦GIV  ES 
GLENN 


♦GO 
♦GO  E? 

♦GONE 
♦GOT 
♦GOTTEN 
♦GOULD 
GRAESSER 
GR  ANDaRAPrDS 
GREEN 
GREEN®  DAY 
GREENVILLE 
GREG 

♦GRENOBLE 
GRIGNETTI 
♦GROUP 
♦HAD 
HAIFA 
HARRIS 
HART 
HARTLEY 
HARVARD 
♦HAS 
HASKINS 
HAUSMANN 
♦HAVE 
HAWAII 
HAWKINS 
♦ HE 

HEALTH 
HEARIN  G 
HEART 
HEDTI.FR 
HENDERSON 
♦HER 
♦HERE 
♦HIM 
♦HIS 
HOLLAN  D 
HOLMDEL 
HONGaKONG 
HONOLULU 
HOT.5SPRINGS 
♦HOUSTON 
♦HOW 
HRA 

HUGGINS 
HUMAN 
HU  MR  EL 
♦HUNDRED 
HUNGARY 


♦HYPOTHETICAL 

♦I 
IBM 
I C A I 
♦ICCL 
IDAHO 

IDAHO!  FALL  0 
♦IEEE 
♦IFIP 
♦IJCAI 
TLL INO IS 
♦IN 
TNDTA 
I NO  I AN  A 
INDIAN  APOL IS 
INEXPENSIVE 

initial 

♦INTERNATIONAL 
IOW  A 
♦IRAO 
♦IS 

1ST 

ISRAEL 
*'  T 

♦ITALY 
♦ITEM 
ITEI1IZE 
ITHACA 
♦JACK 
JACKSON 
J ACKSCNV  ILLE 
JAI  hZ 
JAMAICA 
JAKES 
♦JANUARY 
JAPAN 
♦JERRY 
JERUSALEM 
JIM 
JOAN 
JOE 
JOHAN 

JOHANNESBURG 

♦JOHN 

!OHNS9>  HOPK  INS 

JOHNSON 

JONES 

JPL 

♦JUAREZ 

JUDY 


JULIE 

♦JULY 

♦JUNE 

JUNEAU 

♦JUST 

♦K 

KALAHA  ZOO 
LIKOW 
K,  NSAS 
X A NS  AS  31  CITY 
KEEP 
KEEPS 
KKN 

KENTUCKY 

KENYA 

KEPT 

KEY9WE  ST 
XHUEN 
KIDSTON 
KINGSTON 
♦KLATT 
♦KLOVSTAD 
♦KNEW 
♦KNOW 
♦KNOWN 
♦KNOWS 
KNOXVILLE 
KONA 
KYOTO 
♦L.  A. 

LA2>  PAZ 
LAFAYETTE 
LANCASTER 
LARAMIE 
LAS3VEGAS 
♦LAST 
♦LATE 
♦LAURA 
LAUSANNE 
LAWRENCE 
♦LEAVE 
♦LEAVES 
LEAVITT 
♦LEBANON 
LEEDS 
♦LEFT 
LENI  NG  RAD 
♦LESS 
LEVISON 
LEW  IS 


LIEGE 
LIMA 
LINCOLN 
LI  NCOLN«LABS 
•LINDA 

♦LINGUI  STICS 
LISBON 
LI  SP 
♦LIST 

LI  TTLE a)ROC K 
LOGO 
LOIS 
♦LONDON 
♦LONG 

LOSS)  ALAMOS 
♦L0S3ANGELES 
LOUISIANA 
I.OUI  SVILLE 
I.OV  ETT 
LUKAS 
♦LYN 
♦MADE 
MADELEINE 
MADER 
MADISON 
MADRID 
MAINE 
♦MAKE 
♦M,  TS 
♦MAK*:  JL 
MALAYS  TA 
MAN-MACHINE 
MANCHESTER 
MANILA 
♦MANY 
♦MARCH 
MARIO 
♦MARK 
MARSEILLE 
MARTIN 
MARY 

HARY3ANN 

MARYLAND 

♦MASSACHUSETTS 

MATTHEW 

MAURER 

♦MAY 

♦ME 

MEDFOR  D 
♦MEETING 
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MELBOURNE 
M EM  PHI S 
ME  NLODPAPK 
MERRIAM 
♦MEXICO 
M FX  T CO i? C TTY 
♦MIAMI 
MI  AMI«  BEACH 
MICHAEL 
MICHIGAN 
MI  KE 
MILAN 

♦ MILEAGE 
MILL  EH 

MI LWAUKEE 
MINNEAPOLIS 
MI NN5SOTA 
MINDS 

♦ MI  SCELLANEOIIS 
MISSISSIPP  T 
MI S SOU  LA 

♦ M IS  SOI)  R I 
MIT 

M ITN  ICK 
MNEMO 
MOOT  FY 
♦MONDAY 
*M‘>N  FY 
MONTANA 
MONTESiCARLO 
♦MONTEREY 
MONTGOMERY 
♦MONTH 
MONT  tiF  AL 
♦MOPE 
v-  O H S p 
MOSCOW 
♦MUCH 
MHLI.ARKEY 
MULTIPLY 
MURPHY 
“UR  RAY  OH  ILL 
MUSIC 

♦ MUST 
♦MY 

MY  ER 

NAIROBI 

NASA 

♦ NASH -W  EBBE p 
NASHVILT.F 


N ATO 
NR  S 
NCC 
NCI 

♦NEARLY 

NEBRASKA 

♦ NEED 
NELLRKE 
NEVADA 

♦NEW 

NEWDBRUNSW  ICK 
NEW ®DE  LUX 
NEWDHAMPSHIRR 
NEWDHA VEN 
N FWDJ  ERSEY 
NEW  AME  XICO 
NEW  DOR  LEANS 
♦NEW D YORK 
NEWDYORKDCITY 
NEWD7.E  ALAND 
NEW  PORTDNEWS 

♦ NEXT 
NICF 

♦NICKERSON 
N I N C ns 

NI  NDS 

♦ MINE 

♦ NT  NETEEN 

♦ NIN  FTEENTH 
♦NI NETY 
♦NINTH 

NLS 

♦ NO 
NOME 

♦ NON  E 
NORFOLK 
NORM  AL 

♦NORMALLY 
NURTHDCAROL  INA 
NORTH <AD  AKOT  A 
NO  RW AY 

♦ NOT 

EOT  R ED  DAM E 
♦NOVEMBER 

♦ NOW 
♦NUMBER 

♦ OCTO  RFR 

♦ OF 
OGDEN 

♦ OH 


OH  l 0 

♦ OK 

OK  LAHOMA 
OK  LAUD M ADC  TTY 
OLIVER 
OLYMPI A 
OMAHA 

♦ ON 

♦ ONE 

♦ ON  E-WAY 

♦ ONLY 
ONR 

♦OR 

ORDINARILY 

OREGON 

ORLANDO 

ORLY 

OSLO 

♦OTHER 

♦OTTAWA 

♦OUR 

♦OU’" 

♦OUTSTANDING 

♦OVER 

♦OVER-BUDGET 

♦OVERHEAD 

OXFORD 

PACKET 

PADUCAH 

PAIGE 

PALMDBEACH 

PALMDSPRINGS 
♦PALODALTO 
♦PARIS 
PASADENA 
♦ PAST 
PATRICK 
PAUL 
PEARL 
P EN  DI.FTON 
♦PENNSYLVANI A 
PENS  ACOLA 
♦PEOPLE 
PEORIA 
♦PERDIEM 
PERKINS 
♦PERSON 
P ERT  H 
PERU 
PEW 
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•PHILADELPHIA 
PHI  LIPPI NES 
PHOENIX 
•PHONOLOGICAL 
•PHONOLOGY 
PICKETT 
PI ERFE 
♦PISA 

•PITTSBURGH 
•PLAN 
•PLAN  E 
•PLEASE 
PLUMMFR 
•PLUS 
POCATELLO 
POLAND 
POi TLAND 
PORTUGAL 
POUGHK  EFPSIE 
PRAGUE 
PRECEDING 
PRECISE 
PR  EC  IS  ELV 
PRESCOTT 
PRESENTLY 
PREVIOUS 
PREV  IOUSLY 
PREWETT 
PRICE 
PRINCETON 
•PRINT 
•PROJECT-N 

• PROPOS  ED 
PROVIDENCE 
PROVO 

PUERT09RIC0 

PULLMAN 

PURDUE 

• PURPOS  E 
•PUT 

•PUT-PAST 

• PUTS 
•QUARTER 

QUEBEC 

0UPBEC9CITY 

•QUIT 

RADIO 

RALEIGH 

RAND 

RATIO 


•RAY 

•RECENT 
•RECENTLY 
•RECOGN  IT  10 N 
•RECORD 
REGINA 

• PEG  I ST  RATI  ON 
REGULARLY 

•REMAIN 

• REMAIN  DER 
•REMAINING 

RENO 

REPORT 

• REST 
RETURN 

RHODES  ISLAND 

• RICH 

• RICHAR  D 
RICHMOND 
RIO 

RI0SDE9JANEIR0 
ROANOK  E 
ROBERT 
•ROBOT 
ROCHESTER 
ROLLA 
ROLLINS 
ROME 
RON 

RONALD 

ROSS 

•ROUND-TRIP 

ROIJRKE 

RUMANIA 

RUSSELL 

•RUSSIA 

RUSTY 

RUTGERS 

SACRAMENTO 

SAGAMORE 

SALEM 

•SALT9L AKE 
SA  LT9L AKE9CITY 
S AN®  AN  TON  10 
•SAN9DI EGO 
•S  AN®  PR  ANCISCO 
SAN 900 SE 
S AN® JU AN 
•SANTA9BARBARA 
SANTA®  CR  UZ 


SANTA®  FE 
SANTASMONICA 
SANTIAGO 
SA09PAUL0 
SARAJIAN 
SASKATOON 
•SATURDAY 
SAVANNAH 

e r*  i u rv>  n ■ 
o v,  n ti  i/w  l\  n 

SCHANTZ 
•SCHFDULE 
SCHIPF 
SCHOLAR 
•SCHWARTZ 
SCOTLAND 
SCREENING 
SCRL 
• SDC 
SEATTLE 
SEAWAY 
*S ECOND 
SELECTION 
SELPRIDGE 
•SEND 

SEPARATE-ADJ 
♦SEPTEMBER 
•SEVEN 
•SEVENTEEN 
•SEVENTEENTH 
•SEVENTH 
•SEVENTY 
SEVERAL 
•SHE 
SHEILA 
SHELLY 
SHORT 
•S  HOTJLD 
•SHOW 
♦SHOWN 
SHREVEPORT 
S IL V ER9SPR ING 
•SINCE 
SINGAPORE 
siouxaciTY 
•SIX 

•SIXTEEN 

♦SIXTEENTH 

♦SIXTH 

•SIXTY 

♦SOCIETY 


♦SOME 
♦SOM  EBO  DY 
♦SOMEONE 
SOM  FT  M IN  0 
♦SOMF.TI  ME 
SOON 
SOPH  I F 
SOUK  D 

SOUTHS AFRI CA 
SOUTHSCAROL  IN  A 
SOUTPffiD AKOT  A 
SPAIN 
SPEAR 
♦SPEECH 
♦SPEND 
♦ SPENDS 
♦SPENT 
♦SPLIT 
♦SPLIT- PAST 
♦SPLITS 
SPODICK 
SPOK  AN  E 
♦spRI NG 
SPRINGFIELD 
SRI 

STTLIOHN 
♦STSLOUIS 
STSPAUL 
ST  a)  PETERSBURG 
STSTHOMAS 
STANFORD 
♦ST  APT 
♦STATE 
♦STATUS 
♦STEFRI NG 
ST  EV  F 
STEVENS 
STILLW ATER 
♦STOCKHOLM 
STOCKTON 
STOCK WELL 
♦STOP 
STRESS 
ST ROLLO 
STUDY 
S U DT  R A CT 
SUE 

♦SU  FFIC  IENT 

SUM 

♦S1I  1 M FP 


SUNSVAI.LEY 

♦SUNDAY 

SIINY 

♦SUPPOSE 
♦SU  PROS  TIT’  ION 
♦SUP 

♦SUHPI.U  S 
SURVEY 
SUSAN 
SUSSMA  N 
♦SUTHERLAND 
SWAPTII  MORE 
♦SW  EDEN 
SWITZERLAND 
SY  DNFY 
SYMPOSIUM 
SYRACUSE 
SYSTEM 
t ACOMA 
TAIPEI 
TAIWAN 
♦ TAKE 
*T  AK  FN 
♦TA  KES 
TALLAHASSEE 
TA  MPA 
TAOS 
TED 

TFI.SAV  IV 
♦TELL 
♦TEN 
TFNFX 
T ENN  FS  SFF 
♦TENTH 
TEST 
TESTING 
♦TEXAS 

TEXAS3'CH'’ISTTAN 

THAILAND 

♦THAN 

♦THANK®  YOU 
*TH  A " 

♦T  M r 
♦TH  El R 
*T  H PM 
♦THEN 
♦THERE 
♦THESE 
*”H  EY 
♦THIRD 


♦Till  RTF  FN 
♦Til  1 RTF  FN  v!i 
♦THI  °TI  ET|| 
♦THIRTY 
♦THI  S 
THOM  AS 
TH  ORNTON 
♦THOSE 

♦ THOMSA  ND 
♦THR  EF 
♦THROUGH 
♦THURSDAY 

TICKET 
TIERNAN 
TT  LL 
TIM  FS 
♦TO 
TODAY 
TOKYO 
TOLEDO 
TOM 

TOMLIN  SON 
TOMORPOW 
TONY 
♦TOO 
♦TOOK 
TOPEKA 
TORONTO 
♦^OTAL 
TOULOUSE 
♦TRAIN 
TRANSCRIBER 
♦TRANSCRIPTION 
♦TRAVEL 
TRENTON 
♦TRIP 
TROY 
TUCSON 

♦ TUESDA  Y 
TULSA 

♦TURI N 
♦TW  EL  FT  H 
♦TWELVE 
♦TW  ENTI  ETH 
♦TWENTY 
TW INS  F ALLS 
♦TWO 
TYPICAL 
TYPICA  LLY 
UCLA 


-105- 


.1 


n LI  EH 

♦WHAT 

♦U  N A NTI Cl  PAT  ED 

♦WHEN 

*UM  RUDGETTED 

♦WHERE 

♦UNDER 

♦WHICH 

♦UNDER-  BUDGET 

♦WHITE9PLAI NS 

♦UNDERSTAND I NG 

♦WHO 

♦UNEXPECTED 

WHOLE 

♦UNI  VAC 

♦WHOM 

♦UNTAKEN 

♦WHOSE 

UNTIL 

♦WHY 

♦UPCOMING 

♦WILL 

♦ US 

WILLIAMS 

(JSC 

WI  LLIAMSBURG 

USSR 

WILMINGTON 

♦USUALLY 

WINCHESTER 

♦ UTAH 

WINNIPEG 

UTICA 

WI NSTON3SALEM 

VANCOUVER 

♦WINTER 

VERACRUZ 

♦WI  SCONSIN 

VERIPY 

♦WITH 

VERMONT 

♦WOLF 

VICTOR 

♦WOODS 

VICTOR IA 

WORK 

VIENNA 

♦WORKSHOP 

VIRGIN9ISLANDS 

♦WOULD  , 

VIRGINIA 

WYOMING 

V IR  GIN  IAS B EACH 

X-RAY 

♦VISIT 

YALE 

VISUAL 

♦YEAR 

VOCODER 

♦YES 

VOICE 

YESTERDAY 

WALLA9WALLA 

YORKTOWN 

WALLY 

YOR  KTO  W KSH  E IGHTS 

WALTER 

♦YOU 

♦WANT 

♦YOUR 

WARSAW 

YUGOSLAVIA 

♦WAS 

♦ZERO 

♦WASHINGTON 

ZUE 

WATERLOO 

♦WAY 

♦WE 

♦WEDNESDAY 

♦W  EPK 

WELLINGTON 

♦W  ENT 

♦ WERE 

♦WEST 

WESTSLAPAYETTE 

WESTSP  ALMS  BEACH 

WESTSVIRGI NIA 
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Append  1 x f> 

Dictionary  Kx pans  ion  - A User's  Guide 


In  this  section,  we  Rive  the  sequence  of  terminal  interactions 
involved  in  producing  the  various  dictionary  files  required  for  running  the 


system.  The  reader  without  3 spe 
section  is  invited  to  skip  it. 


oin,-'  technical  need  for  reading  this 


• i.  t a.  w vt 


Dictionary  expansion  generally  begins  by  calling  a subsystem  called 
RULETESTER.3AV,  which  currently  resides  on  directory  [ BBND]<CW00DS> . This 
subsystem  is  a LISP  sysout  that  contains  a set  of  phonological  rules 
already  loaded  with  the  basic  Bobrow-Fraser  rule  tester  and  the  BBN 
extensions .• 


There  are  two  versions  of  the  basic  rule  expansion  function:  EXPLIST 
operates  entirely  in-core  and  is  useful  for  relatively  small  dictionaries. 
We  have  run  dictionaries  up  to  500  words  with  it,  but  there  is  very  little 
extra  room  for  working  storage.  EXPFILE,  the  second  version,  operates 
incrementally  from  a file,  sending  its  output  to  another  file.  The 
description  of  the  former  in  [Woods,  1975c]  is  still  accurate  and  will  not 
be  repeated  here.  The  latter  has  changed  semewhat  since  then,  as  will  be 
described  below.  The  advantage  of  the  in-core  expansion  is  that  the  entire 
dictionary  is  left  in  core  after  the  expansion,  allowing  a phonologist  to 
explore  anything  that  he  finds  strange  in  the  output  summary  by  tracing  a 
re-expansion  using  the  FXPWORD  function.  In  the  incremental  version,  he 
must  first  use  a function  GFTWORD  to  load  into  core  any  word  that  he  wishes 
to  re-expand  before  using  FXPWORD. 


•A  basic  loadup  of  LISP  plus  FLIP  (a  pattern-matching  language  embedded  in 
LISP)  plus  tnc  Bobrow-Fraser  rule  tester  is  saved  In  the  sysout  file 
BASICRUlETESTER .SAV; 1 . which  currently  resides  in  directory  [ BBND] <W00DS> . 
This  sysout  is  built  on  an  ancient  version  of  INTERLISP,  namely, 
LIS  .SAV^.  Bringing  the  rule  tester  up  on  a newer  version  of  INTERLISP 
would  require  a new  version  of  FLIP,  which  would  have  to  be  provided  by 
Warren  Teitelman  at  Xerox  PARC.  The  symbolics  of  the  Bobrow-Fraser  rule 
tester,  on  a lile  [ BBND] <WOODS>PHON  would  also  probably  have  to  be  modified 
to  run  on  a current  LISP.  Symbolics  for  this  file  may  also  live  at  Xerox 
PARC  and  at  Speech  Communications  Research  Laboratory  in  Santa  Barbara. 
The  BBN  extensions  to  the  rule  tester  live  on  the  file  PRULES , which  again 
might  require  some  modifications  to  run  on  a newer  version  of  LISP,  but  I 
believe  the  changes  there  would  not  be  too  difficult.  Since  the  dictionary 
expansions  are  only  run  occasionally  and  our  resources  were  short,  we  have 
not  made  any  attempt  to  bring  up  a newer  version  of  the  rule  expansion 
system.  — W.  Woods. 
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EXPFILE , which  Invokes  incremental  dictionary  expansion  from  a file, 
take,  six  arguments  - FILE,  FULETREE , TFL,  TREEPROP,  VERFLAG,  and 
JACKFLAG.  The  first  is  the  name  of  the  file  to  be  expanded,  the  second  is 
the  name  of  a list  of  rules  to  be  used  for  the  expansion,  and  the  third  1= 

: to  be  set  to  T when  a trace  of  the  individual  rule  match 

attempts  is^deslred  for  rule  debugging  and  testing.  TREEPROP  is  an 
indicator  of  the  property  on  which  the  pronunciation  to  be  expanded  s 
stored  and  Indicates  which  phase  of  the  dictionary  expansion  is  being 
reduested.  (The  dictionary  files  are  stored  in  a symbolic  analog  of  the 

LISP  property  list,  consisting  of  property-name/property-v.lue  pairs  or 

each  word.)  VERFLAG  indicates  whether  a VERDICT  file  for  the  verification 
component  is  desired,  and  JACKFLAG  Indicates  whether  a DICTTEXT  file  for 
the  lexical  retrieval  component  is  desired.  If  the  expansion  is  being  done 
for  testing  of  the  rules  only  and  not  to  produce  input  dictionary  files  for 
the  speech  understanding  system,  these  last  two  arguments  would  be  NIL. 

All  of  the  dictionary  expansion  routines  make  a certain  assumption 
about  the  format  of  the  name  of  the  dictionary  file  to  be  expanded  and  the 
names  of  the  files  that  the,  produce.  Specifically,  assuming  that 

dictionary  to  be  expanded  has  the  ordinary  TENEX  file  name  convention 

FILE.EXTjn  (Where  FILE  is  the  basic  file  name,  EXT  Is  a file  extens  on 
added  to  the  name,  and  n is  version  number  of  the  file),  the  rule  expansion 
system  names  all  of  its  files  with  an  extension  of  FILE-EXTIn , which  i 
constructs  from  the  input  file  name , extension , and  version  number . or 
example,  if  the  dictionary  file  being  expanded  was  named  TRAVELDICT. , (no 

extension),  then  the  expanded  dictionary  would  be  named 

,,  in  ,,qo  tM*-  narticular  file  name  throughout 
EXPDICT. TRAVELDICT-! 19-  We  will  use  thi.  particular 

the  examples  in  this  section. 

in  our  current  set  of  phonological  rules,  there  are  three  lists  of 
rules:  INFLECTLIST,  ZUELIST,  and  APRLIST,  for  the  three  successive  phases 

of  expansion.  The  first  phase  is  performed  by  typing  the  command: 


EXPFILE(TRAVELDICT. ; 1 9 INFLECTLIST  NIL  NIL  T) 


(All  of  the  arguments  to  EXPFILE  beyond  the  last  one  typed  will  be 
defaulted  to  NIL).  This  will  produce  the  following  files: 
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INFLFCTfTD -TRA VEuDICT- ! 19. ; 1 
EXPDICT . TRAVELDICT- ! 19 . ; 1 
EXPHONES. TRAVELDICT-! 19.; 1 
VERDICT. TRAVELDICT-!  19. ; 1 

(assuming  older  versions  of  the  same  file  names  did  not  already  exist). 
The  la3t  is  the  dictionary  file  that  is  used  by  the  Verification  component. 
The  first  is  an  intermediate  file  that  results  from  adding  the  inflected 
words  to  the  original  root  words  and  sorting.  It  can  be  used  for  later 
expansions  without  having  to  go  through  this  process  again.  (For  example, 
if  we  were  to  edit  some  of  the  rules  on  INFLECTLIST  and  want  to  do  the 
expansion  again,  we  could  type: 

EXPFILE( INFLECTED .TR A VELD ICT- ! 19  INFLECTLIST  NIL  TREE  T) 

where  the  argument  TREE  indicates  that  this  is  an  expansion  of  a previously 
expanded  file  and  that  the  property  on  which  the  pronunciations  are  stored 
in  the  file  is  TREE.)  The  EXPDICT  file  produced  is  in  the  appropriate  form 
for  the  Syntactic  and  Control  components  of  the  system  to  use.  The 
EXPHONES  file  contains  a summary  of  the  rule  applications  for  use  by  the 
phonologist . 

The  second  phase  of  expansion,  the  main  phonological  rule  expansion  on 
the  output  of  the  first  phase,  is  initiated  by  typing: 

EXPFILE(EXPDICT.TRAVELDICT-! 19  ZUELIST  NIL  EXPHONES) 

This  indicates  that  th°  pronunciations  to  be  expanded  are  those  that  were 

placed  on  the  EXPHONES  property  by  the  previous  expansion  and  that  the  list 

of  rules  to  use  this  time  is  ZUELIST.  As  a result  of  this  expansion,  the 

following  files  would  be  produced: 

A PRDICT.TR A VELDICT-! 19. ;1 
A PR  PH  ONES . TRAVELDICT-!  19. ; 1 

Thc-3e  files  are  exactly  like  the  EXPDICT  and  EXPHONES  files  that  were 
produced  by  the  previous  expansion,  except  they  they  include  the  effects  of 
the  additional  phonological  rules  in  ZUELIST.  The  APRPHONES  file  produced 
at  this  point  is  for  the  phonologist  to  look  at  for  debugging  rules,  and 
the  APRDICT  file  is  for  input  to  the  next  phase. 
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The  final  APR  expansion  is  initiated  by  typing: 


EXPFI!.E(  APRDICT  .TRAVELDICT- ! 19  APRLIST  NIL  EXPHCNES  NIL  T) 

where  the  final  argument  indicates  that  a DICTTEST  file  for  the  Lexical 

Retrieval  component  is  to  be  made.  This  expansion  will  produce  the  files: 

APRDICT. TRAVELDICT-! 19. ]2 
APRPHONES . TRAVELDICT- ! 19 . J 2 
DICTTEXT. TRAVELDICT-! 19. 

where  DICTTEXT  is  the  file  to  be  used  by  the  Lexical  Retrieval  component, 
and  the  APRDICT  and  APRPHONES  files  are  as  before,  except  that  they  contain 
the  results  of  the  APR-rule  expansion. 
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